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(54) Transmission path controlling apparatus, method and program 



(57) The present invention relates to the distribution 
of load in a plurality of routes from a communication ap- 
paratus which servos as a start point to another com- 
munication apparatus which serves as an end point from 
among communication apparatus in a network so that 
toad distribution of load can be performed between the 
set routes. An apparatus provided in a router in a net- 
work includes a traffic characteristic collection section 
for collecting traffic characteristics of transmission paths 



connected to the router, a traffic characteristic notifica- 
tion section notifies other routers of the collected traffic 
characteristics, a load calculation section calculates 
load based on the collected traffic characteristics, a de- 
cision section decides based on the determined load In- 
formation whether or not a transmission path should be 
added or deleted, and a load equalization section equal- 
izes the determined load among the plurality of trans- 
mission paths. 



FlG.I(b) 



a 

O 
CO 

CO 



6 



Load •quftlixalton 



TranxmUalon 
. Una 

teflon 



3B 



load calculation 



Tr«mc chara statistic* 
to other routers 



Traffic clwractartitU 
(lOtlflcoUcn aactlo* 



Traffic charactarlafc 
racaelloA taction 



Traffic char acladattt 



3A 



Traffic charaotarf atlca 
from other route ra 



LU 



Printed by Jouve. 75001 PARIS (FR) 



REST AVAILABLE COPY 



EP 1 130 849 A2 



Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

[0001] This invention relates to a transmission path controlling apparatus provided in a communication apparatus 
(for example, a router or load equalization apparatus) which forms an network, and further to a transmission path 
controlling method executed by the apparatus as well as a medium on which a transmission path controlling program 
70 used in the apparatus is recorded. The method and apparatus may be provided within, for example, an Internet Protocol 
network (in the following description an Internet Protocol is sometimes referred to simply as IP; accordingly, an Internet 
Protocol network is referred to simply as IP network). 

Description of the Related Art 

15 

[00021 The Internet is intended to handle all communication applications on the internet protocol and is being pop- 
ularized rapidly at present. Further, the Internet originally has a connectionless network architecture which does not 
involve establishment of a connection, and Is routed to an outgoing path based on a destination address descnbed in 
an IP packet. In short, in a communication apparatus having the function described, at a point of time when an IP 
20 packet arrives at the apparatus, the IP packet is merely transferred to an outgoing path which corresponds to the IP 

address. * 
[0003] In the Internet having such a network architecture as described above, in order to determine to which outgoing 
path each IP address should be transferred, a protocol for path determination called routing protocol is handled between 
communication apparatus. 

?5 [0004] At present, as an algorithm for the routing protocol, a system which employs the algorithm of Dijkstra is com- 
monly used. However, the routing protocol mentioned has a problem in that a plurality of routes to a destination cannot 
be selected or the routing protocol is executed only when the network topology varies. 

[0005] In short, with such a technique as described above, once a route is established, an IP packet is transferred 
only along the route, and as a result, there is the possibility that chronic congestion occurs. 
30 [0006] Further, such a technique as described above has a problem also in that, since a plurality of routes cannot 
be set, when a certain route suffers from congestion, even if some other route is unoccupied, this unoccupied route 
cannot be used. 

[0007] Thus, the following techniques have been proposed. 
35 (1-1) Equal cost muttipath technique in the OSPF (Open Shortest Path First) 
(1-2) Multipath technique in the IBGP (Internal Border Gateway Protocol) 

[0008] In particular, the method according to the technique of (1-1) is a method which uses a function of the OSPF, 
40 and uses e function capable of laying a plurality of paths having an equal cost from a communication apparatus which 

serves as a start point to another communication apparatus which serves as an end point to distribute load among the 

plurality of paths (multipaths). ,. 

[0009] Meanwhile, the method according to the technique of (1-2) uses a function of the IBGP lo lay a plurality of 

paths and distribute load among the plurality of paths (multipaths). 
45 [0010] However, the method which uses a function of the OSPF according to (1-1 ) above has the following problems. 

[0011] In particular, there is a restriction in that the multipaths must be equal In cost, and the degree of freedom in 

path selection is low. Further, since the OSPF is used, the method cannot cope with a variation of the network load on 

a real time basis. 

[0012] Also the method which uses a function of the IBGP according to (1-2) above has a restriction in that the 
so multipaths must be equal in cost and has a problem in that the degree of freedom in path selection is low. similarly to 
the problems of the method which uses a function of the OSPF. 

[0013] It is to be noted that also a technique has been proposed wherein a path is set in the layer 3 between data 
processing apparatus which connect networks to each other and load is distributed In a unit of a route in the layer 3 
to dynamically provide a traffic balance to Improve the network performance (refer to the official gazette of Japanese 
55 Patent Laid-Open No. HE1 10-224400). With the technique, however, even if a traffic balance can be provided between 
networks, load distribution in each network cannot be performed. Further, only load distribution in the layer 3 and in a 
unit of a route is taken into consideration, and there is the possibility that load distribution control may be complicated. 
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SUMMARY OF THE INVENTION 

[00141 The present Invention has been made in view of such problems as described above, and it Is an object of the 
present invention to provide a transmission path controlling apparatus and a transmission path controlling method as 
well as a medium on which a transmission path controlling program is recorded, wherein a plurality of routes (trans- 
mission paths) from a communication apparatus (source communication apparatus) which serves as a start point to 
another communication apparatus (destination communication apparatus) which serves as an end point from among 
communication apparatus in a network such that load can be distributed among the set routes so as to allow traffic 
engineering In a network such as the Internet irrespective of the network topology and without depending upon the 
type of the transmission paths. 

[001 51 FIGS 1 (a) 1 (b) and 1 (c) are explanatory view3 of a principle of the present invention. First, a transmiss.on 
path controlling apparatus according to the present invention is used with such a network as shown in FIG. 1(a). the 
network includes a source communication apparatus (an origination communication apparatus) 1S. a destination com- 
munication apparatus 1D, a plurality of transmission paths 2-i (I = 1 to N: N is a natural number) capable of being set 
between the source communication apparatus 1S and the destination communication apparatus 1 D, and an interme- 
diate communication apparatus 1R interposed In any one of the transmission paths 2-i, and provided .n one of the 
communication apparatus 1S. 1D and 1R which form the network (in the following description, where the source com- 
munication apparatus IS, destination communication apparatus 1D and intermediate communication apparatus 1R 
are represented without specific distinction among them, they are referred to merely as communication apparatus 1, 
and the network is an IP network for example). . 
[00161 The transmission path controlling apparatus (the source communication apparatus 1S and the destination 
communication apparatus 1D) according to the present invention includes, as shown in FIG. Kb), a traffic characteristic 
collection section 3, a traffic characteristic notification section 4, a load calculation section 5, a decision section 6 and 
a load equalization section 7. 

[00171 Here the traffic characteristic collection section 3 collects traffic characteristics of those of the transmission 
paths 2-1 connected to the communication apparatuslor the other communication apparatus Jhe traffic^ 
collection section 3 includes a traffic characteristic collection section 3A for collecting traffic character.st.cs at the com- 
munication apparatus 1 itself, and a traffic characteristic reception section 3B for receiving lrafficcharactenst.cs notified 
of from the other communication apparatus. . „ K „ 

[0018] The traffic characteristic notification section 4 notifies the other communication apparatus of the traffic char- 
acteristics collected by the traffic characteristic collection section 3 (particularly 3A). 

[0019] The load calculation section 5 calculates load based on the traffic characteristics collected by the traffic char- 

[0020] , ' C -fhI dwteloTsS™ 6 discriminates whether or not a transmission path should be added or deleted based 
on the load Information determined by the load calculation section S. and the load equalization secl.on 7 equalizes the 
load determined by the load calculation section 5 among the plurality of transmission paths. „-„., 
[0021] The transmission path controlling apparatus (the intermediate communication apparatus 1 R) . » c <=° rd '"9^ *^ 
present Invention includes, as shown in FIG. 1(c), a traffic characteristic collection section 3 and a traffic characteristic 

raS^HeraS traffic characteristic collection section 3 collects traffic characteristics of those of the tiansmission 
path32-i connected to the communication apparatus 1 orthe other communication , apparatus The traffic ^racteristic 
coition section 3 Includes a traffic characteristic collection section 3A for collecting traffic characteristics at the com- 
munSonapparatuslitself.andatraffic charactertetic reception section 3B for receiving traffic charactenstics notified 
of from the other communication apparatus. .„«:„ „k.„, 

[0023] The traffic characteristic notification section 4 notifies the other commun.cat.on apparatus of the traffic char- 
acteristics collected by the traffic characteristic collection section 3 (3A and 3B). 

ESTTl Is to be noted ihat. while H is possible to provide the transmission path controlling apparatus which Is 
provided in each of the source communication apparatus 1S. destination communication apparatus 1D and .nterme- 
dTecomLmc^ 

t ft. transmission path controlling apparatus, which is provided In the source communication W-j-talS. 
w* the traffic characteristic collection section 3, traffic characteristic notification secbon 4. load calculation section 5. 
o^sten section 6 and load equalization section 7 and prov.de, as shown in figure 1(c) the transmission path continuing 
apparatus which Is provided In each of the destination communication apparatus 1D and the inte™ed.ate communl- 
catioiTapparatus 1R other than the source communication apparatus IS. only with the traffic characteristic collection 

section 3 and traffic characteristic notification section 4. ., 

rOMSl It is to be noted that the traffic characteristic collection section 3 may have means for smoothing the co lected 
^characteristic values, or may have means for determining a maximum value of the co.iected traffic characteristic 
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values as a representative value of the traffic characteristic values. 

[0026] Further, the traffic characteristic notification section 4 may have means for notifying (providing) the information 
regarding the traffic characteristics making use of a packet provided by a routing protocol, or may have means for 
notifying the information regarding the traffic characteristics after each of a fixed period, or else may have means for 
5 notifying the Information regarding the traffic characteristic with a message of the RSVP expanded. 

[0027] Further, the traffic characteristic collection section 3 may have means for collecting a unit of a flux of flows 
for collecting traffic characteristics in a unit of a label which is used by a communication apparatus of the label switch 
type such as MPLS (Multi-Protocol Label Switching). 

[0028] The traffic characteristic collection section 3 provided in the source communication apparatus 1S may have 
10 means for discriminating a utilization (a utilization rate) of each of the transmission paths 2-i connected to the source 
communication apparatus 1S based on an average utilization (an average utilization rate) of the transmission paths 
2-i connected to the Intermediate communication apparatus 1R collected from the intermediate communication appa- 
ratus 1R. 

[0029] II is to be noted that the traffic characteristic collection section 3 provided in the source communication ap- 
is paratus 1S may have means for discriminating a utilization of each of the transmission paths 2-I connected to the 
source communication apparatus 1S based on a maximum utilization of the transmission paths 2-l connected to the 
intermediate communication apparatus 1R collected from the intermediate communication apparatus 1R. 
[0030] Further, the traffic characteristic collection section 3 provided in the source communication apparatus 1 S may 
have means for discriminating after each fixed period whether or not each of the transmission paths 2-i is in congestion. 
20 [0031 J The load calculation section 5 provided in the source communication apparatus 1S may have means for 
calculating effective load taking a number of abandoned packets generated in the intermediate communication appa- 
ratus 1R into consideration. In this instance, an effective load calculation unit may be constructed so as to set an upper 
Irmrt to the effective load when the effective load is to be calculated. 

[0032] Furthermore, the transmission path controlling apparatus may be constructed such that the load calculation 
25 section 5 provided in the source communication apparatus 1 S has means for calculating effective load taking a number 
of abandoned packets generated in the intermediate communication apparatus 1R into consideration; and the decision 
section 6 provided in the source communication apparatus 1S has means for calculating an effective bandwidth from 
the effective load obtained by the effective load calculation unit and determining, regarding all of the transmission paths 
as one virtual pipe, a utilization of the virtual pipe; 
30 [0033] Further, the transmission path controlling apparatus may be constructed such that the load calculation section 
5 provided in the source communication apparatus 1S has means for calculating effective load taking a number of 
abandoned packets generated in the intermediate communication apparatus tR into consideration, and the decision 
section 5 provided In the source communication apparatus 1S has a pipe utilization calculation unit for calculating an 
effective bandwidth from the effective load calculated by the effective load calculation unit and determining, regarding 
35 all of the transmission paths 2-i as one virtual pipe, a utilization of the virtual pipe, and comparing the pipe utilization, 
obtained by the pipe utilization calculation unit with a set reference value and determining whether a transmission path 
is to be added or deleted in accordance with a result of the comparison. 

[0034] The decision section 6 provided in the source communication apparatus 1S may be constructed so as to add 
a transmission path when the pipe utilization obtained by the pipe utilization calculation unit is higher than a reference 

40 value for addition of a path, or may be constructed so as to delete a transmission path having a low load when the 
pipe utilization obtained by the pipe utilization calculation unit is lower than a reference value for deletion of a path. 
[0035] The load equalization section 7 provided in the source communication apparatus 1 S may have a move effec- 
tive bandwidth calculation unit for determining an effective bandwidth to be moved with each of the transmission paths 
2-i from an average effective bandwidth calculated for all of the transmission paths 2-1 based on an effective bandwidth 

<5 obtained from effective load obtained taking a number of abandoned packets generated in the intermediate commu- 
nication apparatus 1R into consideration. 

[0036] Further, in this instance, the load equalization section 7 provided in the source communication apparatus 1S 
may distribute packets flowing into the source communication apparatus 1S to the plurality of transmission paths in 
proportion to the effective bandwidths to be moved calculated by the move effective band calculation unit. 
so [0037] It is to be noted that the load equalization section 7 provided in the source communication apparatus 1 S may 
be constructed so as to perform a calculation using a Hash function based on an address and distribute traffic flows 
at random based on a result of the calculation. 

[0036] Then, in the present invention, in a network which Includes a source communication apparatus 1S, a desti- 
nation communication apparatus 1D, a plurality of transmission paths 2-i capable of being set between the source 
55 communication apparatus 1S and the destination communication apparatus ID. and an intermediate communication 
apparatus 1R interposed in any one of the transmission paths 2-i. 

(A1) a traffic characteristic collection step of collecting traffic characteristics of those of the transmission paths 2-i 
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connected to the other communication apparatus, and 

(A2) a traffic characteristic notification step of notifying the source communication apparatus 1S of the traffic char- 
acteristics collected by the traffic characteristic collection step are. performed by communication apparatus other 
than the source communication apparatus of the network, and 
5 (B1) a traffic characteristic collection step of collecting traffic characteristics of those of the transmission paths 

connected to the source communication apparatus 1S, 

(B2) a load calculation step of calculating load based on one or both of the traffic characteristics collected by the 
traffic characteristic collection step and the traffic characteristics obtained from the other communication apparatus. 
(B3) a determining step of determining whether or not a transmission path should be added or deleted based on 
10 the load information determined by the load calculation step, and 

(B4) a load equalization step of equalizing the load determined by the load calculation step among the plurality of 
transmission paths may be performed by the source communication apparatus 1S. 

[0039] Also a medium is disclosed on which a transmission path controlling program for being used by one of Uib 
15 communication apparatus 1 which forms the network shown In FIG: 1 (a) is recorded, the transmission path controlling 
program causing a computer to function as a traffic characteristic collection unit for collecting traffic characteristics of 
those of the transmission paths connected to the communication apparatus or the other communication apparatus, 
traffic characteristic notification unit for notifying the other communication apparatus of the traffic characteristics col- 
lected by the traffic characteristic collection unit, load calculation unit for calculating load based on the traffic charac- 
20 teristics collected by the traffic characteristic collection unit, decision section for deciding whether or not a transmission 
path should be added or deleted based on the load information determined by the load calculation unit and a load 
equalization unit for equalizing the load determined by the load calculation unit among the plurality of transmission 
paths. 

25 BRIEF DESCRIPTION OF THE DRAWINGS 
[0040] 

FIG. 1(a) is en example of a network of the present invention; 
30 FIG. 1(b) is an example of an embodiment of the present invention; 

FIG. 1 (c) is an example of another embodiment of the present Invention. 

FIGS. 2(a) and 2(b) are views for explaining an MPLS network. 

FIG. 3 is a view for explaining a label used in MPLS. 

FIG. 4 is a view for explaining the label used in the MPLS. 
35 FIG. 5 is a view for explaining an LSP. 

FIG. 6 is a view for explaining a flow of a message of the RSVP-LSP-Tunnel. 

FIG. 7 is a view for explaining an MPLS network applied to an embodiment of the present invention. 

FIG 8 is a functional block diagram of an MPLS router as an embodiment of the present Invention. 

FIG. 9 is a functional block diagram showing details of a traffic engineering section of the MPLS router as the 
40 embodiment of the present invention. 

FIG. 10 is a view for explaining an effective load. 

FIG. 11 is a view for explaining an effective bandwidth to be moved. 

FIG. 12 is a view for explaining an example of a manner of load adjustment. 

FIG. 13 is a view for explaining an example of a Hash function. 
45 FIG. 14 is a view for explaining a view of a network for experiment of load adjustment. 

FIG 15 is a view for explaining an example of a result of evaluation of a load adjustment performance. 

FIG* 16 is a view for explaining another example of a result of evaluation of a load adjustment performance. 



50 



55 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0041] In the following, an embodiment of the present invention is described with reference to the drawings. 
[0042] As an embodiment of the present Invention, a case wherein an MPLS router as a router (an example of a 
communication apparatus) of the label switch type is equipped with the present transmission path controlling apparatus 
and a route different from a routing protocol is set using the MPLS (Multi-Protocol Label Switching) and using a label 
function of the MPLS to effect load distribution is described. Here, the MPLS signifies a system which can simply and 
effectively perform a service which is difficult to realize with a router at present by using a label of the layer 2 tn a 
specialized manner for an IP. . 
[0043] In the following, an outline of the MPLS is described first, and then a load distribution method and a load 
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distribution algorithm (Including description of operaUon) of Ihe MPLS are described. 
(2-1) Description of an Outline of the MPLS 

5 [0044] First, an outline of the MPLS is described. FIGS. 2(a) and 2(b) show a network architecture and a manner of 
basic packet transfer of the MPLS. An MPLS network is composed of an edge label switching router (Edge Label 
Switching Router, hereinafter referred to as edge LSR) 11E and a core label switching router (Core Label Switching 
Router, hereinafter referred to as core LSR) 11 C. This Is called MPLS domain. 

10045] Here, the edge LSR 1 1 E is a router which is positioned on the boundary with an existing internet network and 
10 adds a label to a packet to realize a network layer service of a high performance and a high added-value. This edge 
LSR terminates a LAN interface of an unlabeled frame base (Frame Base). 

[0046] Meanwhile/the core LSR 11C is a router which exchanges a label of a labeled packet or ceil. This core LSR 
11C may be constructed so as to support not only label exchange but also layer 3 routing or layer 2 switching. 
[0047] A protocol which is used in cooperation with a routing protocol of a standard network layer to exchange label 

15 information between devices on an internet network by label switching is the label distribution protocol (LDP). 

[0048] It is to be noted that reference characters 9S and 90 In FIG. 2(a) denote each a terminal such as a personal 
computer. . 
[0049] In such an MPLS network as shown in FIG. 2(a). a label coordinated with a path leading to a destination 
terminal is added to an IP packet arriving in the MPLS domain by an edge LSR 11 E. In FIG. 2(b), a label A is provided 

20 to the IP packet, and the IP packet is transferred to a core LSR 11C. The core LSR 11C looks over only the label and 
switches the IP packet to an outgoing path corresponding to the label A. When the packet is transferred to the switched 
outgoing path, a label B is provided newly to the packet. This is called label swapping. In this manner, in the MPLS 
domain while label swapping is performed successively, the IP packet is successively transferred toward a target 
destination terminal 9D in accordance with the label. At the edge LSR 11E at the exit of the MPLS domain, the label 

25 is removed and transferred to the MPLS domain. 

[0050] Subsequently, a label used in the MPLS is described. 

[0051] First, the two following methods are available to realize the MPLS. 

®Method wherein an existing switch is used 
30 ©Method wherein a new label is defined 

[0052] The method according to ©wherein an existing switch is used is a method wherein an existing switch such 
as an asynchronous transfer mode (ATM) or a frame relay (represented also as FR), and in the ATM, a region of a 
virtual path identifier (VPI) and a virtual channel identifier (VCI) is used as a label, but in the frame relay, the DLCI is 
35 used as a label. 

[0053] In the method according to ©wherein a new label is defined, a newly defined label called "shim header Is 
used. The shim header is Inserted between the layer 3 and the layer 2, and an outline of it is illustrated in FIG. 3. 
[0054] To the shim header, a TTL and a pointer S for a label stack are provided in addition to a label as shown in 
FIG 3 The label stack is used for hierarehization of labels, and by using the label stack, it is possible, for example, to 
40 hierarchize a plurality of MPLS domains in an MPLS domain to build up a network. In FIG. 3. where S = 1 . it is recognized 
that this is the last label. 

[0055] It is to be noted that, in addition to labels for the ATM/FR. also labels for the PPP over SONET and labels for 

the ether frame (LAN) are defined (refer to the PPP and the LAN shown on the right side in FIG. 3). 

[0056] Further, a label has a length of. for example. 4 octets and Is indicated as shown in FIG. 4. Such labels can 

«5 have a hierarchical structure, and the structure is called "label stack". The label and other additional information illus- 
trated in FIG. 4 is sometimes indicated in the MPLS header defined by the MPLS or indicated in the L2 header or the 
ATM header. fti 
[0057] The S (Bottom of Stack) indicated in FIG. 4 Is set to 1 where the entry Is the last entry positioned at the 
lowermost position of the label stack, and it Is set to 0 for ail of the other entries. 

50 [0058] Meanwhile, the TTL (Time-to-Live) shown in FIG. 4 represents a field used to encode a TTL and is decre- 
mented one by one before a labeled packet is forwarded. A packet whose TTL becomes zero as a result of the dec- 
rementing must not be forwarded any more, and in this instance, either the packet Is abandoned as It is. or it is handed 
to processing of the network layer with the label stack removed therefrom. This packet must not be forwarded also 
after it is handed to the processing of the network layer. 

55 [0059] Subsequently, a label switched path (LSP) is described. The LSP signifies e path from an LSR at the entrance 
to another LSR at the exit, and may pass intermediate LSRs. an example can be seen also from FIG. 5. A label at 
each LSR Is determined in a coordinated relationship with the LSP. While description is hereinafter given, in the present 
invention, collection of traffic characteristics is performed for each LSP. 
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[0060] Further, a method of establishing an LSP is described. 

[0061] First, as a method of establishing an LSP, the following two systems have been proposed. 

©Method wherein the LDP (Label Distribution Protocol) Is used 
5 ©Method wherein the RSVP-tunnei is used 

[0062] In the following, a method which uses the RSVP-Tunnel of ©is described. 

[0063] The LSP-tunnel Is executed expanding the conventional RSVP. and the background of this system arises 
from that, where the MPLS Incorporates the RSVP, a flow can be managed with increased flexibility by unifying them, 
to [0064] The LSP-tunnel defines, for the RSVP, the following five new objects [LABEL REQUEST (used in e path 
message (PATH message)). LABEL (used in a Resv message). EXPLICIT ROUTE (used in a PATH message). 
RECORD ROUTE (used in a PATH message or a Resv message) and SESSION ATTRIBUTE (used In a PATH mes- 
sage)]. 

[0065] Here, the LABEL REQUEST object is used In a PATH message and is used to determine a label regarding 
is the path by an intermediate router and a reception router. In order to establish a path (= LSP-tunnel), a source LSR 
produces a PATH message having a LABEL REQUEST, and further, a node which has transmitted the LABEL RE- 
QUEST must make preparations for reception of a Resv message corresponding to the PATH message. If a PATH 
message which does not include a LABEL REQUEST is received, then a LABEL REQUEST object must not be written 
into the PATH message. 

20 [0066] While a LABEL object is used In a Resv message, in the LSP-tunnel. a label is placed In the Resv message 
and returned to the source. Then, the Resv message is sent back toward the source along a route of the PATH message. 
Further, each node which receives the Resv message having the LABEL object uses the label as a label of the output 
side of the LSP-tunnel, and if the node is not the source of the LABEL REQUEST, then the node executes processing 
of allocating a new label and placing the new label into the LABEL object of the Resv message. Then, the label trans- 

26 mltted to the upstream is used as an Input side label of the LSP-tunnel. 

[0067] An EXPLICIT ROUTE object (ERO) is used in a PATH message. If this object appears in a PATH message, 
then the PATH message must be transferred along a path explicitly indicated by the ERO. The ERO is intended to be 
used for explicit routing. The RECORD ROUTE object is used in a PATH message or a Resv message and allows a 
transmission node to receive information regarding an actual path of the LSP-tunnel. This is similar to a path vector 

30 (path vector) and can be used also for detection of a loop. 

[0068] A SESSION ATTRIBUTE object is used in a PATH message. This SESSION ATTRIBUTE object can be pro- 
vided to a PATH message and used for identification and diagnosis of a session. Further, this object has pre-emissfon 
(preemption), priority, local protection and so forth. 

[0069] FIG. 6 illustrates a flow of a message of an LSP-tunnel in which the RSVP is used. The RSVP-tunnel is a 
35 Downstream-on-demand system since a label is allocated from the downstream. As can be recognized also from FIG. 
6, a first MPLS router (LSR1) in an MPLS domain places a LABEL REQUEST object into a PATH message of the 
RSVP and transmits the PATH message to the downstream. Thereafter, if the PATH message arrives at a node (LSR4) 
of the destination, then the destination node (LSR4) places a LABEL object into a Resv message and returns the Resv 
message to the upstream. The manner of allocation of a label is such as described hereinabove. 
<o [0070] In the present invention, a traffic characteristic value is embedded into and transferred together with the RSVP 
message. 

(2-2) Description of the Load Distribution Method in the MPLS 

45 [0071] In order to perform load distribution using the MPLS, using the LSP described hereinabove, a plurality of 
transmission paths [path, I.e., LSP; in the present embodiment, a path and an LSP are used in the same meanings] 
12-i (i = 1 to N: N is a natural number) are set between a source MPLS router (a source communication apparatus) 
11S and a destination MPLS router (a destination communication apparatus) 11Q as shown in FIG. 7. In this instance, 
the LSP is set using a label of the MPLS- For the setting of the LSP, an LSP setting protocol such as the RSVP-LSP- 

50 Tunnel Is used. It Is to be noted that, in FIG. 7, reference character 11 R denotes a repeating MPLS router (an inter- 
mediate communication apparatus). 

[0072] Further, a portion of a transmission path between routers is called link, and, for example, a transmission path 
12-1 in FIG. 7 is composed of M links (link_1 to llnk_M). 

[0073] it can also be recognized that the network architecture of the MPLS shown in FIG. 7 may be an IP network 
55 which includes a source MPLS router 11 S. a destination MPLS router 11 D. a plurality of transmission paths 12-i (i = 1 
to N: N is a natural number) which can be set between the source MPLS router 11S and the destination MPLS router 
11D and a repeating MPLS router 11R interposed In one of the transmission paths 12-i. It is to be noted that, also in 
the present embodiment, where the source MPLS router 11S, destination MPLS router 11D and repeating MPLS router 
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11 R are represented without specific distinction among them, they are referred to merely as MPLS routers 11 . 
[0074] FIG. 8 shows a functional block diagram of an exemplary embodiment of an MPLS router which forms an IP 
network. As shown in FIG. 8, the MPLS router 11 includes an IP packet transfer section 1 11 , a routing protocol section 
112, a path selection section 113, an LSP selection section 114. an LSP setting section 115 and a traffic engineering 
5 section 116. 

[0075] Here, the IP packet transfer section 111 has a function of performing desired processing such as IP packet 
transfer processing for an input packet and transferring the packet to another router. The routing protocol section 112 
mediates notification of traffic characteristics to another router or collection of traffic characteristics from another router- 
It is to be noted that the routing protocol section 112 further has a function of rendering, when the MPLS router is 
10 connected to the network, the routing protocol operative to discover the other MPLS routers connected to the network. 
For the discovery, a Hello packet is used usually. 

[0076] The path selection section 113 has a function of producing a link state database (Link State Database) 113A 
for path selection when the topology of the network Is discriminated by the routing protocol. The link state database 
113A cooperates with the LSP setting section 115 through the LSP selection section 114 to set an LSP with another 
15 MPLS router. 

[0077] The LSP selection section 114 has a function of cooperating with the link stale database 113A of the path 
selection section 113 to produce an alternate path information database 114A for LSP selection. At this time, a routing 
table unique to the MPLS which is not used in conventional routing protocols is produced. The alternate path information 
database 1 14A cooperates with the traffic engineering section 116 to set an alternative path using an LSP as needed. 
20 In short, a plurality of LSPs to an MPLS router at the end are set so that load may be balanced between the LSPs. 

[0078] The LSP setting section 115 has a function of setting a plurality of LSPs to an MPLS router at the end in 
response to a result of LSP selection by the LSP selection section 114. A result of the LSP setting by the LSP setting 
section 115 is transmitted to the IP packet transfer section 111, and a transfer form of an IP packet is set by the IP 
packet transfer section 111 . 

25 [0079] The traffic engineering section 116 has a series of functions regarding a load balance according to the present 
Invention. As shown in FIG. 9 r the traffic engineering section 116 includes a load observation section 116A, a traffic 
characteristic value calculation section 116B. a traffic engineering calculation section 116C and a load adjustment 
section 11 6D. 

[0080] The load observation section 116A has a function of sending traffic characteristic values such as a utilization 
30 and a packet loss number sent thereto from the packet transfer section 111 to the traffic characteristic value calculation 
section 11 6B. The traffic characteristic value calculation section 11 6B has a function of performing smoothing or max- 
imization of the traffic characteristic values and transmitting a resulting value to another MPLS router and another 
function of collecting traffic characteristic values sent thereto from another MPLS router and sending them to the traffic * 
engineering calculation section 116C. 
35 [0081] The traffic engineering calculation section 1 16C has a function of performing calculation of an effective load, 
an effective bandwidth and so forth and performing adjustment of load in response 1o the bandwidlhs of the individual 
LSPs. A result of the calculation (operation) obtained by the traffic engineering calculation section 116C is transmitted 
to the load adjustment section 11 6D. 

[0082] The load adjustment section 116D has a function of determining a width of a traffic share based on the infor- 
40 mation from the traffic engineering calculation section 116C and sending a result of the determination to the LSP setting 
section 115 through the LSP selection section 114. The LSP setting section 115 thereafter sets a traffic share width at 
the IP packet transfer section 111 . 

[0083] In particular, the traffic engineering section 116 has functions of a traffic characteristic collection section for 
collecting traffic characteristics of LSPs (transmission paths) connected to the LSP router itself or the other MPLS 

45 routers, a traffic characteristic notification section for notifying the other MPLS routers of the traffic characteristics 
collected by the traffic characteristic collection section, a load calculation section for calculating load based on the 
traffic characteristics collected by the traffic characteristic collection section, a decision section for deciding whether 
or not an LSP should be added or deleted based on the load information determined by the load calculation section, 
and a load equalization section for equalizing me load determined by the load calculation section among the plurality 

50 of LSPi (i is a natural number). This traffic engineering section 116 makes a principal part of the transmission path 
controlling apparatus of the present invention. 

[0084] It is to be noted that principally the load observation section 116A and the traffic characteristic value calculation 
section 11 6B exhibit the function of the traffic characteristic collection section 3, the traffic engineering section 116 
exhibits the function of the traffic characteristic notification section 4, and the traffic engineering calculation section 
ss 116C exhibits the functions of the load calculation section 5. decision section 6 and load equalization section 7. 

[0085] Further, in order to allow an MPLS router to have the functions of the traffic characteristic collection section, 
traffic characteristic notification, load calculation section, decision section and load equalization described above, nor- 
mally from a medium on which a transmission path controlling program for causing a computer to function as traffic 
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characteristic collection unit for functioning as the traffic characteristic correction section described above (the traffic 
characteristic collection unit has a function of collecting traffic characteristics of those of the transmission paths con- 
nected to the MPLS router or the other MPLS routers), traffic characteristic notification unit for functioning as the traffic 
characteristic notification section (the traffic characteristic notification unit has a function of notifying the other MPLS 

a routers of the traffic characteristics collected by the traffic characteristic collection unit), a load calculation unit for 
functioning as the load calculation section (the load calculation unit has a function of calculating load based on the 
traffic characteristics collected by the traffic characteristic collection unit), a discrimination unit for functioning as the 
decision section (the discrimination unit has a function of discriminating whether or not a transmission path should be 
added or deleted based on the load information determined by the load calculation unit), and a load equalization unit 

to for functioning as the load equalization section (the load equalization unit has a function of equalizing the load deter- 
mined by the load calculation unit among the plurality of transmission paths) Is recorded, the transmission path con- 
trolling program is installed Into each of the MPLS routers. ^ ^ ^. 
[0086] The Installation may be performed by setting a recording medium such as a floppy disk or an MO disk to a 
computer or by using a transmission path controlling program distributed over a network (for example an IP network). 

is [0087] It is to be noted that, since an actual IP network may be formed like a cobweb using an MPLS router as a 
node, any MPLS router can serve as a source MPLS router, a destination MPLS router or a repeating MPLS router, 
and consequently, although the present embodiment is shown as an example wherein the traffic engineering section 
116 of each of the source MPLS router 11S. destination MPLS router 11 D and repeating MPLS router 11 R shown in 
FIG 7 has all of the functions of the traffic characteristic collection section, traffic characteristic notification section, 

20 load calculation section, decision section and load equalization section described above, it is otherwise possible that 
the traffic engineering section of the source MPLS router 11S has the functions of the traffic characteristic collection 
section traffic characteristic reception section, load calculation section, decision section and load equalization, and 
the traffic engineering section provided in each of the destination MPLS router 11 D and repeating MPLS router 11 R 
other than the source MPLS router 1 1S has only the functions of the traffic characteristic collection section and traffic 

25 characteristic notification section. 

[0088] In the following, a load distribution algorithm performed in the traffic engineering section 116 is described in 

detail. 



30 



(2-3) Load Distribution Algorithm Performed in Each MPLS router 11 

[0089] First the MPLS router 11 performs, for each port thereof, traffic collection regarding required traffic items [for 
example an output packet amount from an output port, an abandoned packet number (NLOSS) at the output port, a 
logical bandwidth (LBW) of the output port and so forth] as traffic characteristic values after each time tl [seconds (sec)] 

35 [OOM^^en? the MPLS router 1 1 calculates a transmission line utilization CUTY at present from the obtained traffic 
characteristic values in accordance with the following expression (procedure 2). 

CUTY = output packet amount from output 
40 port/logical bandwidth of output port 0) 

[0091] Hence it follows that the traffic characteristic collection section has means for calculating utilizations of the 
transmission paths connected to the router based on the collected traffic characteristic values. As a result, in what 
45 condition each of the transmission paths Is used can be grasped in detail. 

[0082] At this time, also working processing of information (the transmission line utilization CUTY) regarding the 
obtained traffic characteristic values is performed. 
[0093] The processing manner is such as follows. 

[0094] ©Smoothing processing Is performed. Such smoolhing processing is performed, as an example, by calcu- 
so lating the following expression each time an observation value is obtained to obtain a value MUTY 

MUTY = a x CUTY + (1 - a) x MUTY ( 2 ) 

55 where a is a smoothing coefficient. 

[0095] Consequently, it is considered that the traffic characteristic collection section has means for smoothing mfor- 
mation (the transmission line utilization CUTY) regarding collected traffic characteristic values. More particularly, it is 
considered that the traffic characteristic collection section has means for smoothing, after traffic charactenstlcs of 
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transmission paths connected to the MPLS router are collected, the statistical values in accordance with, for example, 
the Exponential Moving Average method. As a result, an Influence of a sudden traffic variation can be reduced. 
[00961 Further, since the transmission path utilization CUTY at present determined from the obta.ned traffic charac- 
teristic values is smoothed as described above. It follows that a smoothing unit may be constructed so as to smooth 
the utilizations calculated by the utilization calculating unit. Consequently, the utilizations of the individual transmission 
paths can be grasped accurately while eliminating traffic variation influences. 

[0097] ©Maximum value extraction processing is performed. Such maximum value extraction processing is per- 
formed by calculating a maximum value of observation values. In other words, the maximum value of the observation 
values is placed as the work value MUTY. This may be represented in an expression as follows. 

MUTY « mex(observation values) ( 3 ) 

[00981 Consequently, it is considered that the traffic characteristic collection section has means for determining a 
maximum value of collected characteristic values as a representative value of the traffic characteristic values. As a 
result transmission path selection control can be performed within a degree of certainty. 

[0099] It is to be noted that, as regards the smoothing processing and the maximum value extraction processing 
described above, both of the smoothing processing and the maximum value selection processing may be performed 
or only one of the smoothing processing and the maximum value selection processing may be performed. 
[0100] ®A total number TLOSS of abandoned packets may be calculated in accordance with the following expres- 

rai 011 -aOSS(n) = TLOSS(n-l) + NLOSS where TLOSS(n) is the total number of abandoned packets after updated. 
TLOSS(n-1 ) is the total number of abandoned packets before updated, and NLOSS is the number of newly abandoned 

\0W2] Thereafter, the MPLS router 11 executes distribution of the track characteristic values (MUTY, TLOSS and 

so forth) after each time t2 [sec] making use of Flooding of the routing protocol. 

[0103] It is to be noted that TLOSS is reset (that is. to 0) after completion of the flooding. 

[0104] Also where the maximum value is adopted for MUTY. MUTY Is reset (that Is, to 0). 

[0105] Consequently. It is considered that the traffic characteristic notification section has means for notifying infor- 
mation regarding traffic characteristics utilizing a packet prepared by the routing protocol. More particularly, it is con- 
sidered that the traffic characteristic notification section has means for distributing, after traffic characteristics of the 
transmission paths connected to the MPLS router are collected, the characteristic values making use of a packet 
prepared by the routing protocol such as the OSPF or the IBGP to notify another MPLS router of the information. As 
a result, an existing protocol can be utilized, and consequently. It Is not necessary to newly prepare a routing protocol 
for exclusive use, which contributes to reduction of the cost and so forth. 

[0100] Also it is considered that the traffic characteristic notification section has means for notifying ■nformation re- 
garding traffic characteristics after each fixed period 12 [sec]. As a result, the arithmetic load applied to an arithmetic 
unit such as a CPU which executes arithmetic operation and so forth can be reduced. 

[01 07) Further in the present embodiment, it is considered that the traffic characteristic notification section has means 
for notifying information regarding traffic characteristics with a message of the RSVP expanded. More particularly, it is 
considered that the traffic characteristic notification section has means for distributing, after traffic characteristics of 
the transmission paths connected to the MPLS router are collected, the characteristic values with a message of the 
RSVP expanded to notify another MPLS router of the information. As a result, an existing protocol can be utilized, and 
consequently, it is not necessary to newly prepare a routing protocol for exclusive use, which contributes to reduction 
of the cost and so forth. 

[0108] Further it is considered that the traffic characteristic collection section has means for collecting a unit of a 
flux of flows for collection of traffic characteristics in a unit of a label used In the MPLS. As a result, collection of fine 
traffic characteristic becomes possible, which contributes to augmentation of the collection accuracy. 
[0109] It is to be noted that similar effects can be achieved also by an alternative construction wherein calculation 
of the transmission path utilizations at present is performed after some or all of the smoothing processing, the maximum 
value extraction processing and the total abandoned packet number calculation processing described above are per- 
formed for the collected traffic characteristic values. 

[01 10} On the other hand, also It is possible to employ a further construction wherein the calculation processing of 
the transmission path utilizations is assigned to the MPLS router 11S which serves as a start point of load distribution 
while the repeating MPLS routers and the destination router perform only the other processing (smoothing processing, 
maximum value extraction processing, total abandoned packet number calculation processing and traffic charactenstic 
value notification processing) than the transmission path utilization calculation processing. 

[0111] A load distribution algorithm performed by a traffic engineering section 116' of each of the repeating MPLS 
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routers 11R* and the destination router 11D'(the routers mentioned are hereinafter denoted generally by 11') having 
the construction just described is described in detail below. 

[01 12J First, similarly to the traffic engineering section 116 described above, also the traffic engineering section 116' 
of the MPLS router 11* performs, for each port thereof, traffic collection regarding required traffic items [for example 
5 an output packet amount (L) from an output port, an abandoned packet number (NLOSS) at the output port, a logical 
bandwidth (LBW) of the output port and so forth] as traffic characteristic values after each time tl (seconds (sec)]. 
[0113] Then, the traffic engineering section 116' performs smoothing processing of the obtained output packet 
amounts. Such smoothing processing is performed by obtaining a value ML by calculating the following expression 
each time an observation value is obtained: 

to 

ML = a x L + (1 - a) X ML' (5) 



whore a Is a smoothing coefficient, end ML* is ML calculated in the preceding cycle. 
15 [01 14] Then, maximization extraction processing is performed. This processing is performed by placing a maximum 
value of the observation values as a work value ML. It is to be noted that the maximization extraction processing can 

be omitted. • *-« 

[01 15) Further, similarly to the traffic engineering section 116 described above, the traffic engineenng section 116 
calculates a total number TLOSS of abandoned packets in accordance with the expression (4). 
20 [01 16] Thereafter, the traffic engineering section 116' executes distribution of the traffic characteristic values (ML, 
TLOSS and so forth) after each time t2 (sec] making use of Flooding of the routing protocol. 

[01 17] It is to be noted that, similarly to the traffic engineering section 116, the traffic engineering section 116* resets 
TLOSS (that is, to 0) after each completion of the flooding and. where the maximum value is adopted as ML, the traffic 
engineering section 1 1 6' resets ML (that is, to 0) as well. 

25 [0118] Consequently, similarly to the MPLS router 1 1 described hereinabove, also the MPLS router 11* of the present 
modification can achieve those effects provided by the smoothing processing, the maximum value extraction process- 
ing the total abandoned packet number calculation processing and the traffic characteristic value notification process- 
ing! Further, since the smoothing processing of information regarding the traffic characteristics is assigned to the MPLS 
router 1 1S which serves as a start point of load distribution, the time required to work the traffic characteristics by the 

30 MPLS router 1V can be reduced from that by the MPLS router 11 described above and the MPLS router IV can be 
simplified in construction, which contributes to reduction of the cost and so forth. 

(2-4) Load Distribution Algorithm Performed by the MPLS Router 11S Serving as a Start Point of Load Distribution 

35 [0119] First flooding Information of which the MPLS router 11 S which is determined as a start point of load distribution 
is notified includes the average link utilization (or maximum value of observation values) MUTY (or average output 
packet amount ML), abandoned packet number TLOSS, output port logical bandwidth LBW and so forth as descnbed 
hereinabove. 

[0120] Here where the MPSL router 11S receives the average output packet amount ML, the link utilization MUTY 
40 of each router' is calculated using an output port logical bandwidth LBW and so forth received together. Whether the 
other intermediate routers have an average link utilization calculation function and issues a notification of the calculated 
average link utilization MUTY, the link utilization calculation function need not to be provided for the MPLS router 11 S 

[0121] The MPLS router 1 1 S which serves as the start point of load variation performs path state checking, calculation 
45 of an effective load and load adjustment. 

[01 22] As regards the path state Checking, congestion state checking of the individual routes from the MPLS router 
(start point router) 11 S to the MPLS router (end point router) 11D is performed in the following manner after each fixed 
period 13 [sec]. 

[0123] On the premise that a plurality of paths are laid between the MPLS router 1 1 S which servers as the start point 
so of load distribution control and the MPLS router 11 D which is determined as an end point of the load distnbution control 
and each path extends through a plurality of repeating MPLS routers 1 1 R. the congestion state checking for each route 
discriminates, from utilizations of each path, collected from each of the repeating MPLS routers 11 R of the path, based 
on an average utilization of the transmission paths (links) connected to the repeating MPLS router 1 1 R [this congestion 
state checking manner is referred to as congestion state checking manner where utilizations of links j (link J) of a path 
55 i are averaged]. 

[0124] This can be represented In such an expression as given below. 
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p path I = Average(MUTY(link j, path i) ( 6 ) 

[0125] Consequently. In this instance, it Is considered that the traffic characteristic collection section provided in the 
source MPLS router 11 S has means for discriminating a utilization of each transmission path connected to the source 
MPLS router based on an average utilization of the transmission paths connected to repeating MPLS routers collected 
from the repeating MPLS routers. As a result, this can contribute to augmentation of the efficiency in use of the network. 
[01 26] Further, on a premise similar to that in the congestion state checking manner where utilizations of links j of a 
path i are averaged described above, the congestion state checking for each route may be such congestion state 
checking that it discriminates, from utilizations of each path, collected from each of the repeating MPLS routers, based 
on a maximum utilization of the transmission paths (links) connected to the repeating MPLS router (this congestion 
state checking manner is referred to as congestion state checking manner where a maximum value of utilizations of 
all links of a path I is adopted). 

[0127] This can be represented in the following expression. 

path i = Max(MUTY(link j. path I) (7) 

[0128] Consequently, in this instance, it Is considered that the traffic characteristic collection section provided In a 
source MPLS router has means for discriminating a utilization of each transmission path connected to the source MPLS 
router based on a maximum utilization of transmission paths connected to repeating MPLS routers collected from the 
repeating MPLS routers. As a result, transmission path selection control can be performed within a degree of certainty. 
[0129] It is to be noted that the sum total TLOSS path i of abandoned packets of the path I may be determined In 
accordance with the following expression. 

TLOSS path i = ZTLOSS link j (8) 
[01 30] It is to be noted that i above is ihe number of the path (LSP) and j is the link number. 

[0131] Also » is considered that the traffic characteristic collection section provided in a source MPLS router has 
means for discriminating after each fixed period t3 [sec] whether or not each transmission path is in congestion. By 
this, the arithmetic load applied to an arithmetic unit such as a CPU which executes arithmetic operation and so forth 
can be reduced. 

[0132] Calculation of the effective load (Effective load) p effective path I may be performed in the following manner. 
[0133] Here the effective load is defined as an effective utilization calculated from a utilization of a link and a packet 
loss number at the link. Originally, an actual load applied to the link should be measured. However, the inside of a 
router actually has a multi-stage switch construction and it is difficult to directly measure the load. Accordingly, in the 
present embodiment, a load is estimated using only a utilization of a link and a packet loss number, and the estimated 
load is referred to as effective load. 

[0134] The effective load is described with reference to FIG. 10. In particular, as can be seen apparently from a 
characteristic of FIG. 10. If no packel loss Is involved, then the load p palh i and the effective load at the link are equal 
to each other However, if packet loss occurs, then the load p path i is corrected with a predetermined function f (TLOSS 
path i) so that a rather high load may be obtained, and a resulting value is used as the effective load. It Is to be noted 
that an upper limit p ceiling is provided to the effective load. This can be represented in the following expression. 

p effective path i = ppath i x f (TLOSS path i) (9) 

p effective path I = Min (p effective palh i, p ceiling) where f(TLOSS path i) Is a function regarding TLOSS path I, and 
p ceiling is an upper limit load, that is, an upper limit value to the effective load p effective path i. 
[0135] This function is exhibited in the load calculation section provided in a source MPLS router for calculating an 
effective load taking a number of abandoned packets generated in a repeating MPLS router into consideration. Con- 
sequently, in a situation that a plurality of paths are laid between an MPLS router which is determined as a start point 
of load distribution control and another MPLS router which is determined as an end point of the load distribution control 
and each path extends through a plurality of repeating MPLS routers, when the MPLS router which serves as the start 
point of the load distribution control calculates a load based on traffic characteristics notified of. it can calculate, taking 
the number TLOSS path I of abandoned packets generated in each repeating MPLS router into consideration, the 
effective load p effective path i in accordance wHh the following equation. 
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effective toad to Ith path = load to Ith path x f(abandoned packet number) 

It is to be noted that f (abandoned packet number) is a function wherein the abandoned packet number is a variable. 

5 [0136) As a result, even if a load is not directly measured, an effective load can be estimated by a simple method. 
[0137] Further, in another embodiment, il is considered that the effective load calculation unit is constructed such 
that when it calculates an effective load, it sets an upper limit to the effective load. Consequently, load estimation of 
an excessive amount can be avoided, and as a result, appropriate transmission path selection control can be performed. 
[0138] The load adjustment Is performed in the following manner. 

10 [0139] The toad adjustmenl is executed after each time t4 [sec], and If the effective load to each path (LSP) can be 
calculated in such a manner as described above, then a utilization at all paths (LSP) is determined subsequently. In 
the present embodiment, all paths are regarded bs a single virtual pipe, and a load to the pipe is calculated. 
[0140] In particular, in this instance, it Is considered that the decision section provided in a source MPLS router has 
means for calculating effective bandwidth* from effective loads obtained by the effective load calculation unit and 

15 regarding all of the transmission paths as one virtual pipe and then determining a utilization of the virtual pipe. In this 
instance, the utilization may be determined in accordance with the following expression. 

utilization p effective an when all paths are 
20 regarded as one virtual pipe - L ( p effective path i x LBW 

path i)/£LBW path i 

25 where p effective path is the effective load of the path i, and LBW path i is the logical bandwidth of the path i. 

[0141] Consequently, it can be prevented to unnecessarily add a new transmission path, and this can contribute also 
to effective utilization of the network. 

[0142] Subsequently, effectiveness of multi-paths is verified. . 

[0143] In particular, the utilization obtained from the expression (11) is compared with a set reference value p offerl 
30 for path addition, and when the obtained value of the utilization is higher than the reference value for path addition, a 

new transmission path is added. Further, the utilization obtained from the expression (11) is compared w.th a set 

reference value for path deletion similarly, and if the obtained value of the utilization is lower than a reference value p 

offer2 for path deletion, then a path whose load Is low is deleted. In particular, if the calculated load described above 

is high, then a new path Is added, but if it is low, an existing path is deleted. 
35 [0144] This can be logically represented in the following manner. 

[0145] If peffective all ^ p offerl then 

If timerl was expired then (the left signifies "if the time has expired already") 
search new path and add new path (the left signifies -to the alternative path search processing-) 
40 endif 

elself p effective ah < p offer2 then 

if timer2 was expired then (the left signifies "the timer has expired already-) 
delete path (the left signifies "to the alternative path deletion processing") 
endif 

45 endif 

[0146] Consequently, It is considered that the decision section provided in a source MPLS router has means for 
comparing a pipe utilization obtained by the pipe utilization calculation unit with a set reference value lo perform dis- 
crimination of whether or not a transmission path should be added or deleted in response to a result of the 

so end adds a new transmission path if the pipe utilization obtained by the pipe utilize ■ on calculation unit is h*her han 
a reference value for palh addition, but deletes a transmission path whose load is low if the pipe utilization obtained 
by the pipe utilization calculation unit is lower than a reference value for path deletion. As a result, up-to-date optimum 
path selection can be performed. ^«„~h«h 
f01471 It is to be noted that, as a modification to the verification procedure of the validity of multi-patts descnbed 

55 above it is otherwise possible to calculate, in addition to the pipe utilization (total utilization of all paths) obtained using 
the expression (11). a rate of change of the pipe utilization per time and perform decision of addition or deletion of a 
transmission path based on the rate of change of the pipe utilization. A procedure in this instance is described in detail 
below. 
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[0148] First, a difference value AU(t) between two pipe utilizations obtained after a very small Interval 14 is calculated 
as the rate of change of the pipe utilization per time in accordance with the following expression. = 

5 Au(t) = { id effective all(t-l) — 

p effective all(t)}/t4 ~ (12) 

10 where p effective all(t) is the total utilization of all paths at time t, and peffective all(M) is the total utilization at a 
measurement time M prior by one cycle to time t. 

[0149] It Is to be noted that it is otherwise possible to represent a change of the pipe utilization with respect to time 
obtained in accordance with the expression (11) as a function of the time and calculate a differential value of the pipe 
utilization at a certain point of time and then use the differential value as the rate of change of the pipe utilization per 

is time. This allows a more accurate rate of change of the pipe utilization to be grasped. 

[0150] Then, the utilization obtained in accordance with the expression (11) and a reference value p offerl for path 
addition set therefore are compared with each other, and the rete of change obtained in accordance with the expression 
(12) and a reference value p offerdl for path addition set therefore are compared with each other. Then, when the 
values of the obtained utilization and rate of change both exceed the corresponding reference values for path addition. 

20 a new transmission path Is added. Further, the utilization obtained in accordance with the expression (11) and a ref- 
erence value p offer2 for path deletion set therefore are compared with each other, and the rate of change obtained in 
accordance with the expression (1 2) and a reference value p offerd2 for path deletion set therefore are compared with 
each other Then when the value of the obtained utilization orrate of change Is smaller than the corresponding reference 
value for path deletion, a path which has a comparatively low load is deleted. In other words, also in this instance, if 

25 the load calculated as above is high, then a new path is added, but if the load is low, then an existing path is deleted. 
[0151] This can be logically represented in the following manner. 
[0152] If (p effective all S p offerl or AU(t) * p offerdl) then 

if limerl was expired then (the left signifies "if the timer has expired already") 
30 search new path and add new path (the left signifies "to the alternative path search processing") 

endif 

elseif (p effective all < p offer2 and AU(t) < p offerd2) then 
if timer2 was expired then (the left signifies "the timer has expired already") 
delete path (the left signifies "to the alternative path deletion processing") 
35 endif 
endif 

[0152] Through the construction described above, since decision of addition or deletion of a path is performed with 
a change in time of the pipe utilization taken Into consideration, addition or deletion of a path can be performed more 

40 effectively than where such decision is based only on the pipe utilization. 

[0154] It is to be noted that, while, in the construction described above, logical ANDing (and) of results of comparison 
is used for discrimination of addition of a path and logical ORIng (or) of results of comparison is used for discrimination 
of deletion of a path, the criteria for addition/deletion of a path are not limited to the combination given above, but those 
criteria (logical ANDing and logical ORing) may be used in a different combination in accordance with a configuration 

45 or a state of the network, or some other criteria may be used. This allows an integrated amount of the pipe utilization 
to be grasped appropriately In accordance with the configuration or the state of the network. 

[01 55] Meanwhile, as another modification to the verification procedure of the validity of multi-paths, also it Is possible 
to calculate. In addition to the pipe utilization (total utilization of all paths) obtained using the expression (10). an integral 
amount of the pipe utilization In a fixed time and perform discrimination of addition or deletion of a transmission path 
50 based on the integral amounts of the pipe utilization. 

[0156] In this instance, as an integral amount of the pipe utilization in a fixed time, an integral value J MU(t) of the 
pipe utilization within a fixed time t4 may be calculated in accordance with the following expression. 
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[Expression 1] 
[0167] 



jMU(t)-£'*' 4 MU(t)dt ... (13) 



10 [0158J In other word3. jMU(t) is an integral value of the pipe utilization MU within the fixed time from t to t4. 

[0159] Also It Is possible to calculate a number of limes by which the pipe utilization exceeds a certain reference 
value within the fixed period of time and use the number of times as the integrated amount of the pipe utilization within 
the fixed period of time. This allows the integrated amount of the pipe utilization to be grasped by a simpler construction. 
[01 60] Then the utilization obtained in accordance with the expression (1 1 ) and a reference value p offerl for path 

is addition set therefore are compared with each other, and the integrated amount obtained in accordance wilh the ex- 
pression <1 3) and a reference value p offeril for path addition set therefore are compared with each other Then, a new 
transmission path is added based on results of the comparison. Further, the utilization obtained In accordance with the 
expression (11) and a reference value p offer2 for path deletion set therefore are compared with each other, and the 
integrated amount obtained in accordance with the expression (13) and a reference value p offeri2 for path deletion 

20 set therefore are compared with each other Then, a path which has a comparatively low load is deleted based on 
results of the comparison. 

[0161] Also in this instance, upon decision of addition or deletion of a path, various cntena such as logical ORing 
and logical ANDing of the results of comparison may be used in a suitable combination in accordance with the config- 
uration or the State of the network, and this allows the integrated amount of Ihe pipe utilization to be grasped appro- 
25 priatety. Further, where discrimination is performed based only on the integrated amount without using the utilization, 
decision of addition or deletion of a path can be performed with a simpler construction. 

[0162] From the foregoing construction, it follows that the decision section provided In the source router has a cal- 
culation unit for calculating an effective bandwidth from the effective load obtained by the effective load calculation unit 
and for determining, regarding all of the transmission paths as one virtual pipe, Information regarding a utilization of 

30 the virtual pipe (a pipe utilization, a rate of change of the pipe utilization, an integreted amount of the pipe utilization) 
and is constructed so as to perform decision of whether or not a transmission path should be added or deleted based 
on the Information regarding the utilization of the virtual pipe (at least one of the pipe utilization, the rate of change of 
the pipe utilization, and the integrated amount of the pipe utilization). Consequently, it is possible to perform addition 
and deletion of a transmission path appropriately taking the loads of the transmission paths at present into consider-. 

35 atlon, and distribution of the load can be anticipated efficiently and selection of an appropriate path can be performed 
In accordance with the situation of the network or the performance of the hardware. 

T01 631 Further, similarly to the foregoing construction, it follows that the decision section provided in the source router 
is constructed so as to perform comparison of at least one of a set of the pipe utilization obtained by the calculation 
unit and a first reference value (p offerl. p offer2), another set of the rate of change obtained by the calculation unit 

40 and a second reference value (p offerdl . p offerd2) and a further set of the integrated value obtained by the calculation 
unit and a third reference value (p offeril, p offeri2), and perform decision of whether or not a transmission path should 
be added or deleted In response to a result of the comparison. More particularly, when at least one of the utilization, 
the rate of change and the integrated amount exceeds a corresponding reference value for path addition, it rs deeded 
that a new transmission path should be added, but when at least one of the utilization, the rate of change and the 

45 integrated amount Is lower than a corresponding reference value for path deletion, it is decided that a transmission 
path whose load is comparatively low should be deleted. Consequently, up-to-date optimum path selection can be 
performed in accordance with the situation of the network or the performance of the hardware. 
[0164] By the way if it is decided upon verification of the validity of multi-paths described above that a transmission 
oath should be deleted, then a path of a candidate for deletion is selected based on several conditions, and the utilization 

50 of all paths (loads to the other transmission paths) when the path of the candidate for deletion is deleted is estimated 
and compared with a predetermined reference value. Then, a path to be deleted actually is determined based on a 

result of the comparison. 

[0165] More particularly, deletion of multi-paths is performed in accordance with the following algorithm. 

55 ©That one of all paths which has been laid first is excluded from the object of deletion. 

©That one of all paths which has the smallest usable bandwidth is selected. 

©If a plurality of paths satisly the condition of ®(have the smallest usable bandwidth), then that one of the paths 
which has the lowest effective load is selected. 
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®lf a plurality of paths satisfy the conditions of ©and ©(have the smallest usable bandwidth and the lowest 
effective load), then mat one of the paths which Has the greatest hop.number Is selected. 
S f a plS of paths satisfy the conditions of @. ©and ©(have the smallest usable bandwidth, the lowest 
fZcSZ7 0 f>4 me greatest hop number), then that one of the paths which has been added last « se ected 
S path selected through ©©is determined as a candidate for deletion, and the utilization of the enure mult,- 
naths whan the path is virtually deleted is calculated. _ 
®^e uSzaJoVof me entire mufti-paths after the deletion of me deteUon candidate path calculated ,n © « .com- 
Sr^wtthacongestton discrim.nat.on mrashotd value sal therefore. If ^f^«*»^"£Z2%^ 
L deletion is equal to or higher man the threshold value, then a new deletion cand.data path is selected through 
fiVOaualn from among me paths other than the deletion candidate path. 

Sf ma coToarison in ©reveals that the utilization of ma entire multi-paths after the deletion ,s lower than the 
congestion discrimination threshold value, then the deletion candidate path is deleted actually. 

T01661 Bv oerforming deletion of a path in accordance with such en algorithm as described above it follows mat the 
Eon S^St me soume router includes means for selecting a candidate of a transmission path as an 
oojd of de^n based on me load information determined by the load calculation ^^Zl ^Z^^ 
?hi transmission oaths when me transmission path of the candidate of deletion selected by me deletion candidate 

e!ec£o^ 

unKwfth a predetermined reference value and determining whether or not me transmlss.on path of me cand.date of 
rteiftKon should be deleted in response to a result of the comparison. 

fo eT IU i to be noted mat. when the result of comparison between the information (utilization, rate of change. 
E£L! It) regarding the pipe utilization (total uti.ization of all paths) described he ?^° Ve *^^^ 
vefue is linked to the determination that a transmission path should be delated (for example, when one of the Afferent 
Mnds of information reading me pipe utilization Is lower than the corresponding reference value), the path deleuon 
afconmr^ 

mm' Fromthe construction described above, It follows matthe decision section provided i"^™™^^" 
Includes digger unit for triggering the selection unit, estimation unit and determination unrt to operate based on the 
information regarding the pipe utilization obtained by the calculation unit. 

[0169] ^ Consequently, a fransmission pat h can be deleted in an up-to-date manner in response to a situation of the 

fo^rSma validity of multi-paths is verified in this manner, an affective bandwidth (EBW) to be moved is calcu- 

Stlll in particular, when to equalize load between a plurality of transmission paths added. In order to Olstrlbut© the 
load in proportion to the logical bandwidths of me paths, the effective bandwidth is calculated ,n accordance with an 
expression (14) to perform load distribution. 

effective bandwidth to be moved of ith path = 
(average value of effective bandwidths of all paths 
- effective bandwidth of ith path) 
x logical bandwidth of ith path < 14 ) 

[0172] As a result, balancing of the load can be performed between different paths, and a conc ^ dla ^^ 
trating an effective bandwidth is Illustrated in FIG. 11 . Referring to FIG. 11 , aEBWI indicates an effectrve bandwidth to 
b^ moved and this effective bandwidth is obtained by multiplying an effective .oad by the logical bandwidth LBW. As 
a result, AEBW can be calculated using an expression (15). 

AEBW path I = (p effective all - p effective path 

i x LBW path i < 15 > 

[0173] Consequently, it is considered that the load equalization provided in a source MPLS router has move ^effective 
bandwidth calculation means for determining an effective bandwidth to be moved at each of transm.ss.on paths.from 
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an average effective bandwidth determined for all transmission paths based on effective bandwidths obtained from 
effective loads obtained taking numbers of abandoned packets generated in repeating MPLS routers into consideration. 
Consequently, equalization of load with bandwidths of transmission paths taken into consideration can be performed. 
[0174] Then, after the effective bandwidth AEBW is determined in such a manner as described above, adjustment 
of the traffic share is performed finally. 

[0175] In particular, IP packets flowing in are distributed to a plurality of paths in proportion to effective bandwidths 
to be moved calculated in accordance with the expression (14). When the IP packets flowing In are to be distributed 
to a plurality of paths, calculation by a Hash function Is performed based on IP addresses and the IP traffic flow Is 
distributed at random based on a result of the calculation. 

[01761 The foregoing is described In more detail in regard to a case wherein a CRC (cyclic redundancy check) cal- 
culation is used as the Hash function. . . 

amount Ats path i of traffic to be moved of ith path 
= effective bandwidth to be moved of ith path x Gr x 
TS/(total of (effective load to ith path x logical 
bandwidth of ith path) for all path) 
= AEBW path i x Gr x TS/Z ( p effective path i x LBW 

pathi) < 16 ) 

where Gr is the load adjustment coefficient (1/p ceiling SGri 1/100). TS is the width of a result of calculation of CRC 
arithmetic (In the case of CRC 16. the width of TS is 0-65535). 

Gn load adjustment coefficient 

TS: width of traffic share = range of result of CRC 16 = 65535 

[0177] When load adjustment Is performed in accordance with the expression above, the load adjustment distribu- 
tions before and after the load adjustment are such as illustrated in FIG. 12. 

[0178] In the present embodiment, the track share which has a significance of load adjustment Is a parameter used 
to perform load distribution, and the sum total of traffic shares is a total value of actual traffic through all paths. The 
width of a traffic share is an integer from 0 to 65.535 where the CRC (cyclic redundancy check) 16 is used. When an 
IP packet arrives at a router, the load adjustment apparatus can perform CRC calculation based on an IP address (host 
address, destination address) of the IP packet, and as a result, a certain value between 0 and 65.535 is allocated to 
the IP now In short, the present load adjustment mechanism has a function of degrading an IP address to a width of 
a certain Integer. In other words. IP flows arriving at a router are accommodated at random into Individual paths. 
[0179} Consequently, it is considered that the load equalization section provided in a source MPLS router is con- 
structed such that it has a unit for distributing packets flowing Into the source MPLS router to a plurality of transmission 
paths in proportion to effective bandwidths to be moved determined by the move effective bandwidth calculation unit 
and further that it thereupon performs arithmetic operation using a Hash function based on addresses and ^tributes 
traffic flows at random based on a result of the arithmetic operation. As a result, traffic flows can be distnbuted precisely 
by simple calculation. ^ , . „■ , 

[0180] It is to be noted that, as a simpler and higher-speed Hash function, replacement of posrt.ons and log.cal 
operation may be performed for bit values of an Input value to produce a random integer value. 
[01811 For example, as illustrated in FIG. 13, a destination IP address of 32 bits .ncluded in an IP packet s first 
divided into four areas of 8 bits. Then, the values of two adjacent 8-bits are replaced alternately, and the upper 1 6 bits 
and the lower 16 bits are logically exclusively ORed. Finally, the value of 16 bits produced in this manner .s converted 
into an integer, and the integer is used as a hash value (output value of the Hash function). 

[01821 It to to be noted that the replacement of the positions and the logical operation for the b.t values of the Input 
value are not limited to those described above, and various methods can be used in an arbitrary comb inatlon for Omm. 
[01831 From the foregoing, it follows that the load equalization section provided In the source router .s construe ed 
so as to perform at .east one of replacement of positions and logica! operation for bit values of an mput value wr the 
calculation by the Hash function to produce a random integer value. This allows simpl.ficat.on of the hash calculation 
and also allows distribution of traffic flows to be performed at a high speed. 



17 



15 



20 



25 



EP 1 130 849 A2 

f0184] It is to bo noted that, in order to augment the efficiency of the hash calculation described above, as an Input 
value for the Hash function, a control value (for example, a protocol ID. a port number or the like) other than the address 
included in each packet or the like may be used in addition to the address (host address, destination address) included 
in the packet. In particular, the load equalization section provided in the source router is constructed so as to additionally 
5 use as an input value for the Hash function, a control value other than the address included in each packet This 
increases the disturbance degree of the output value by the hash calculation and allows efficient and effective disbv 
bution of traffic flows. 

[0185] In the following, an example of load distribution of the present algorithm is descnbed. 
w (A) Calculation Example 1 

rol 861 If it is assumed that the logical bandwidths LBW [bit/s] of the paths t . 2 and 3 are 10M. 8M and 2M. respectively 
ari Se JII^^P«flhc^ are 0.5. 0.2 and 0.3. respectively, then the actual traffics [bit/a] of the paths 1 . 2 and 
3 are 1 0M x 0.5 = 5M, 8M x 0.2 = 1 -6M, and 2M x 0.3 = 0.6M. respectively. 

[01 87] Further, if all paths in this instance are regarded as one virtual pipe, then an average utilization Pave effective 
is determined in the following manner. 

pave effective = (5M ♦1.6M + 0.6M)/(10M + 8M + 2M) = 0.36 

[0188] Then, if the effective bandwidth (EBW) [bit/s] to be moved is calculated for the paths 1. 2 and 3. then the 
following results are obtained. 

[0189] In particular, the effective bandwidth A EBW PATH1 to be moved of the path 1 is (0.36 - 0.5) X 10M - -1 .4M. 
he effective bandwidth AEBW PATH2 to be moved of the path 2 is (0.36 - 0.2) x 8M - +1.2BM: and the effec rve 
oTnowidthAEBW 

bandwidths to be moved of all of the paths is calculated, then the result is -1.4M ♦ 1.2.8M + 0.12M - 0. 
[0190] Adjustment of the traffic share is performed based on the calculation result. 

[0191 in particular, for the path 1. 5M - 1.4M « 3.6M; for the path 2. 1.6M + 1.28M = 2.88M; and for the path 3. 0^M 
♦ 0.12M = 0.72M. Consequently, for the paths 1 , 2 and 3, the traffic share should be changed at the ratio of 3.6 x Gr : 
30 2.88 x Gr : 1.2 x Gr. 

(B) Calculation Example 2 

r0192l If it is assumed that the logical bandwidths LBW [bit/s] of the paths 1. 2. 3 and 4 are 150M. 45M. 150M and 
600M respectively, and further the effective loads p effective are 1.2, 1.5 i -)-^ 0 -^~^ B f^^^? d *"J 
traffics [bit/7] of the paths 1. 2, 3 and 4 are 150M x 1.2 « 180M. 45M x1.5 = 67.5M, 150M x 1.1 - 165M. and 600M 

x 0.8 - 480M, respectively. j » ^i na A 

[0193] Further, if all paths are regarded as one virtual pipe, then an average utilization Pave effective is determined 

in the following manner. 

pave effective = (180M + 67.5M «■ 165M + 480M)/(150M + 45M + 150M + 600M) = 892.5/945 = 0.94 
[0194] Then, if the effective bandwidth (EBW) to be moved is calculated for all of the paths 1 to 4. then the following 

[0195] a 7n partSar, the effective bandwidth AEBW PATH1 to be moved of the path 1 is (0.94 - 1 flx150M = -39M; 
the effective bandwidth AEBW PATH2 to be moved of the path 2 is (0.94 - 1.5) X 45M = ^f^^J^^^^ 
AEBW PATH3 to be moved of the path 3 is (0.94 - 1.1) x 150M = -24M, and the effective bandwidth AEBW PATH1 to 
be moved of the path 4 is (0.94 - 0.8) X 600M = 84M. 
so f0196] Adjustment of the traffic share is performed based on the calculation result. 

[0197^ in particular, for the path 1. 180M - 39M = 141M; for the path 2. 67.5M - 25.2M . 42.3M; for the path 3. 16SM 
- 24M = 141M; and for the path 4. 480M * 84M = 564M. Consequently, for the paths 1. 2. 3 and.4, the traffic share 
should be changed at the ratio of 141 x Gr : 42.3 x Gr : 141 x Gr : 564 X Gr. 

55 (2-5) Description of Operation 

[019B] With such a construction as described above, if the MPLS router 11 is first connected to the network, then 
the routing protocol operates and performs discovery of the other MPLS routers connected to the network. For such 
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discovery, usually a hello packet Is usBd. If the lopology of the network is cleared up by the routing protocol, then a 
link state database 113A is produced by the path selection section 113. LSPs are set between the MPLS router 11 and 
the other MPLS routers by the LSP setting section 115 through the LSP selection section 114 using the link state 
database 1 1 3A. Further, the LSP selection section 1 1 4 cooperates with the link state database 1 1 3A of the path selection 
5 section 113 to produce an alternate path information database 114A and thus produces a routing table unique to the 
MPLS and not used by any other conventional routing protocol. The alternate path information database 114A coop- 
erates with the traffic engineering section 116 to set an alternative path using the LSPs es needed. In short, by setting 
a plurality of LSPs between the MPLS router 11 and an MPLS router at the end point, balancing of load can be performed 

between the LSPs. r _ 

10 [0199] It Is to be noted that, while the series of functions regarding balancing of load are functions of the traffic 
engineering section 116. to the load observation section 116A of the traffic engineering section 116 having the con- 
struction just described, track characteristic values such as a utilization and a packet loss number are sent from the 
IP packet transfer section 111. The load observation section 116A sends the values to the traffic characteristic value 
calculation section 116B. The traffic characteristic value calculation section 116B performs smoothing or maximization 

is of the traffic characteristic values and transmits a resulting value to the other MPLS routers. On the other hand, traffic 
characteristic values sent to the MPLS router 11 from the other MPLS routers are collected by the traffic characteristic 
value calculation section 11 6B and sent to the traffic engineering calculation section 116C. The traffic engineenng 
calculation section 11 6C performs calculation of effective loads and effective bandwidths and performs load adjustment 
In accordance with the bandwidlhs of the individual LSPs. 

20 [0200] A result of the toad adjustment is transmitted to the load adjustment section 116D, by which a width of the 
track share Is determined. Then, a result of the determination is set by the IP packet transfer section 111 through the 
LSP setting section 115. • . 

[0201] In this manner, each of the routers 11 R and 11D other than the source MPLS router 11 S collects traffic char- 
acteristics of transmission paths connected to the other routers and notifies the source router of the collected track 

25 characteristics. Meanwhile, the source MPLS router 11S collects traffic characteristics of transmission paths connected 
to the source router and calculates effective loads based on one or both of the thus collected traffic charactenst.es and 
traffic characteristics obtained from the other routers. Then, based on the effective load information, the source MPLS 
router 11S performs discrimination of whether or not a transmission path should be added or deleted and further per- 
forms equalization of the obtained effective load between a plurality of transmission paths. Consequently, each router 

30 has a traffic characteristic observation function and a traffic characteristic value notification function, and a router 
(source router) which serves as e start point of load distribution can dynamically perform bandwidth management of 
a plurality of paths to another router (destination router) which serves as an end point. As a result, load distribution m 
accordance with free bandwidths can be performed. In particular, since a plurality or routes (transmission paths) can 
be set from a router (source router) which serves as a start point to another router (destination router) which serves 

35 as an end point from among routers in an IP network and distribution of load can be performed among the set routes 
traffic engineering can be achieved within the network such as the internet irrespective of the network topology and 
without depending upon the type of the transmission paths. 
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(2-6) Examples of Experiment 



[02021 Evaluation of the load distribution performance by the load distribution apparatus of the present invention i was 
performed using a network construction for experiment shown in FIG. 14. It Is to be noted that, In FIG. 14. LSR 120R- 
1 to 120R-5 denote each an LSR which includes the load distribution apparatus of the present invention, and a load 
generating apparatus 121 is connected to the LSR 120R-1 which serves as a source LSR while a load i™"""B 
<5 apparatus 122 is connected to the LSR 120R-5 which serves as a load destination LSR. A plurality of links Imk1 to 
Iink5 are laid between the LSRs 120R-1 to 120R-5 such that they form a plurality of transmission paths 123-1 and 
1 23-2. in other words, also the network construction for experiment shown in FIG. 14 can be considered as an example 
of the network construction of the MPLS shown in FIG. 7. . ... . 

[0203] In order to confirm operation of the path search function by each of the LSRs 120R-1 to 120R-5 load d,stn- 
so button was performed using the network construction for experiment shown in FIG. 14 and the relat.onsh.p be^een 
the trafficmovementamountand the traffic convergence time with regard to the links linkl and I,nk3 (which correspond 
to the entrances of the plurality of transmission paths 123-1 and 123-2. respectively) connected to the source LSR 
1 20R-1 was evaluated as the load distribution performance by the load distribution apparatus of the present invention. 
Results of the evaluation are illustrated In FIGS. 15 and 16. 
ss S In RG 15. variations in time of the link utilization of the links linkl and Iink2 when load distribution was per- 
formed in the conditions of the input traffic of 50 Mbit/sec (93 flows), the flooding interval (OPSF flooding .ntervaf) by 
IheOSPF of 10 sec, the congestion threshold of 30 Mblt/secand the Hash granularity of 100 % and the .ink uhhzations 
were calculated after each 1 0 seconds are shown in graph. 
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[02051 fmmedlately after the experiment was started, the link utilization (indicated by a void circle in the graph) of 
the link llnkl was a mile lower than 50 Mbit/sec and higher than 30 Mbit/sec of the congestion discrimination threshold 
value, and congestion occurred. On the other hand, the link utilization (a void triangle) of the link Iink3 was almost 0 
Mbit/sec, and it can be seen that one-sldedness of the load appeared between the two links linki end Iink3. 
[0206] The load distribution by the load distribution apparatus of the present invention was started at 40 seconds 
after the experiment was started. The link utilization of the link linki decreased suddenly while the link utilization of the 
link link3 increased suddenly, and it can be seen that the load was distributed between the two links linki and Dnk3. 
Then at 20 seconds after the load distribution was started (at 70 seconds after the experiment was started) . the traffics 
of the two links llnkl and Iink3 were turned and an over-controlled state was entered once. However, while oscillations 
were repeated the difference became very small and the difference between the traffics of the two links linki and Ilnk3 
gradually converged. The convergence time of the traffics was 120 seconds after the load distribution was started. 
10207] On the other hand, in FIG. 16. the experiment was performed in the same conditions as those of FIG. 15 
except that the Hash granularity was changed to 20 %. As a result, although the convergence time of the traffics was 
200 seconds after the load distribution was started and was longer than that of the result Illustrated In FIG. 15. the 
traffics were not turned and an over-controlled state was not entered, but the difference between the traffics of the two 
links Hnk1 and Iink3 converged asymptotically. Thus. It can be seen that distribution of the load was achieved in a stable 
state 

[0208] In particular, if the Hash granularity is set rather high, then although the traffics are rendered unstable through 
over-control load distribution can be achieved in a short lime. In contrast, if the Hash granularity Is set rather low. then 
although the time required for load distribution becomes long, load distribution can be achieved in a stable state while 
preventing the traffics from being rendered unstable through over-control. In other words, appropriate setting of the 
Hash granularity in accordance with the state of the network or the performance of the hardware allows efficient load 
distribution. 

(2-7) Others 

[0209] It is to be noted that, while it is described in the embodiment described above that, using MPLS (Multi-Protocol 
Label Switching) and using a label function of the MPLS, a route different from a routing protocol is set to perform load 
distribution the present invention can be applied even if some other technique is used wherein, using an ATM (Asyn- 
chronous Transfer Mode) , an FR (Frame Relay) or the like and using a protocol unique to the layer 2. a route different 
from the layer 3 (IP) is set to perform load distribution, or using source routing In the IP, a route different from a path 
of a routing protocol is set to perform load distribution. 

[0210] It is also to be noted that, while a router is used as an example of the communication apparatus in the em- 
bodiment described above, the present invention is not only applied to a router but Is widely applicable to other kinds 
of communication apparatus such as a gateway. 

[021 1 ] Further, the present Invention is not limited to the embodiment described hereinabove and can be carried out 
in various forms without departing from the spirit of the present invention. 

(2-8) Addition 

[0212] As described in detail above, according to the present invention, each communication apparatus has a traffic 
characteristic observation function and a traffic characteristic value notification function, and a communication appa- 
ratus (source communication apparatus) which serves as a start point of load distribution can dynamically perform 
bandwidth management of a plurality of paths to another communication apparatus (destination communication appa- 
ratus) which serves as an end point. As a result, load distribution in accordance with free bandwidths can be performed. 
In particular, since a plurality of routes (transmission paths) can be set from a communication apparatus (source com- 
munication apparatus) which serves as a start point to another communication apparatus (destination communication 
apparatus) which serves as an end point from among communication apparatus In an IP network and distribution of 
load can be performed among the set routes, traffic engineering can be achieved within the network such as the Internet 
irrespective of the network topology and without depending upon the type of the transmission paths. 
[0213] The traffic characteristic collection section may smooth information regarding the collected traffic character- 
istic values, and this can reduce an influence of a sudden traffic variation. 

[0214] Further, the traffic characteristic collection section may determine utilizations of the transmission paths con- 
nected to the communication apparatus based on the collected traffic characteristic values. This allows the utilizations 
of the transmission paths to be grasped particularly. In this instance, where a smoothing unit is constructed so as to 
smooth the utilizations determined by a utilization discriminating unit, the utilizations of the transmission paths can be 
grasped accurately while eliminating an influence of the traffic variation. Also where the traffic characteristic collection 
section has determines utilizations of the transmission paths connected to the communication apparatus based on the 
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traffic characteristic values smoothed by the smoothing unit, a similar effect can be achieved. 

10215] Further if the traffic characteristic collection section determines a maximum value of the collected traffic char- 
acteristic values'as a representative value of the traffic characteristic values, then transmission path selection control 
can be performed with a degree of certainty. 

[021 61 Further the traffic characteristic notification section may notify (provide) information regarding the traffic char- 
acteristics making use of a packet provided by a routing protocol, and according to tills construction, an existing protocol 
can be utilized, and consequently, it is not necessary to newly prepare a routing protocol for exclusive use. which 
contributes to reduction of the cost and so forth. ■ . L . ... ... . «= K -«,^ 

102171 Furthermore the traffic characteristic notification section may notify information regarding the traffic charac- 
teristics after each fixed period, and according to this construction, arithmetic load applied to an arithmetic unit such 
as a CPU which executes arithmetic operation and so forth can be reduced. 

[0218] Further, the traffic characteristic notification section may notify Information regarding the traffic charorferishc 
with a message of the RSVP expanded, and according to this construct, an existing protocol can be ut,lized. Conse- 
quent*^ necessary to newfy.preparo a routing protocol for exclusive Use. which contnbutes to rodudon of the 

roSaT 1 FuiSLore. the traffic characteristic collection section may collect a unit of a flux of flows for collecting traffic 
characteristics ih a unit of a label which is used by a communication apparatus of the label switch type, and according 
this construction, traffic characteristics can be collected finely, which contributes to augmentation of the collection 

[O^Ol^Further, the traffic characteristic collection section provided in the source communication apparatus may de- 
termine a utilization of each of the transmission paths based on an average utilization of the transmission paths, and 
this can contribute to augmentation of the efficiency in use of the network. 

[02211 Furthermore, the traffic characteristic collection section provided in the source communication apparatus may 
determine a utilization of each of the transmission paths based on a maximum utilization of the transmission paths, 
and according to this construction, transmission path selection control can be performed with certainty. 
[0222] Further, the traffic characteristic collection section provided in the source communication apparatus may de- 
enrtne after each fixed period whether or not each of the transmission paths is congested, and according to th.s 
SScC arithmetic load applied to an arithmetic unit such as a CPU which executes arithmet.c operation and so 

SaT^urthem^ore. the load calculation section provided in the source communication apparatus may calculate 
SSL i number of abandoned packets generated In the intermediate communication^ 

consideration, and according to this construction, even If a load Is not directly measured, an effecbve toad .can Jbe 
estimated by a simple method. In this instance, an upper limit to the effective load can be set an* I* *" 
construction load estimation of an excessive amount can be avoided, and as a result, appropriate transm«sion path 

l^T^Z £S£r£L provided in the source communication apparatus may ca.cu.aja an effective 
bandwidth from ihe effective ioad obtained by an effective load calculation unit ■ ^ the 

transmission paths as one virtual pipe, a utilization of the virtual pipe, and according to th.s construction it can be 
prevent to unnecessarily add a new transmission path and this can contribute also to effective utilization of the 

jSf ' Furthermore, the transmission path controlling apparatus may be constructed so as to d « t8 ^«*^ er « 
ttansmission path is to be added or deleted based on the information regarding the pipe util.zal.on obtained by he 
ScutationunrThisallows addition and deletion ofatransmissionpathappropriateVwiththa loads to thetransm^sion 
STpraTl taken Into consideration, and distribution of the load can be achieved efficiently. In 
transmission path addition-deletion discrimination unit may be constructed so as to perfonn domination of whettw 
oTnTa transmission path should be added or deleted based on at least one of the pipe utilization, rate of change and 
r m eTrateTamount obtained by the ca.cu.ation means, an appropriate path can be selected In accordance with the 
situation of the network or the performance of the hardware. • 
W226] Further the decision section can be constructed so as to compare the pipe uhl,zat,on. rate of change and 
ntearatedvaTurobtained by the calculation unit with ^pectivereference values and perfom, determination of whether 
oTnora ^nsnnission path should be added or deleted in response ,o a resuff of the comparison, and according to th« 
construction, up-to-date optimum path selection can be performed. 

V227} Furthermore, the calculation un.t may be constructed so as to calculate a difference 

utilizations and/or a differential value of the pipe utilization as the rate of change of the pipe utinzatlon or may be 

SSSd so as to determine a number of times by which the pipe utilization exceeds a reference value or an .nigral 

S of .he P^ Utilization as the integrated amount of the pipe utilization. This allows the calculate amount to be 

changedTn a^rdance with the situation of me network or the performance of the hardware anrt allows efficient path 

selection. 
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[0228] Further, the decision section may selecting a candidate of a transmission path as en object of deletion based 
on the load information determined by the load calculation section, estimating the load to tho other transmission paths 
when the transmission path of the selected candidate of deletion selected is deleted, and comparing thej estimated 
load to the other transmission paths with a predetermined reference value and determining whether or not the trans- 
mission path of the candidate of deletion should be deleted in response to a result of the comparison. This allows 
appropriate selection of a transmission path to be deleted taking en influence when any transmission path' is deleted 
into consideration. In this instance, where the Information regarding the pipe utilization obtained by the calculation unit 
is used as e trigger for the operation, a transmission path can be deleted efficiently in response to the situation of the 
entire network. 

[0229] Further, the load equalization section provided in the source communication apparatus may have a move 
effective bandwidth calculation unit for determining an effective bandwidth to be moved with each of the transmission 
paths from an average effective bandwidth calculated for all of the transmission paths based on an effective bandwidth 
obtained from effective load obtained taking a number of abandoned packets generated in the intermediate commu- 
nication apparatus into consideration, and according to this construction, equalization of load with bandwidths of trans- 
mission paths taken into consideration can be performed. 

[0230] Furthermore, the load equalization section provided In the source communication apparatus may distribute 
packets flowing into the source communication apparatus to the plurality of transmission paths in proportion to the 
effective bandwidths to be moved calculated by the move effective band calculation unit, and according to this con- 
struction, precise equalization of load can be performed. 

[0231] Further, the load equalization section provided in the source communication apparatus may be constructed 
so as to perform a calculation using a Hash function based on an address and distribute traffic flows at random based 
on a result of the calculation, and according to this construction, traffic flows can be distributed precisely by simple 
calculation. 

[0232] In this instance, the load equalization section may be conslructed so as to perform the calculation usmg a 
CRC (cyclic redundancy check) as the Hash function or perform calculation to produce a random integer value by 
replacement of bit positions of an input value as the Hash function or the load equalization section may be constructed 
so as to additionally use a control value other than the address included in each packet as an Input value for the Hash 
function. This allows efficient and effective distribution of traffic flows to be performed. 

Claims 

1. A transmission path controlling apparalus for use with an internet protocol network which includes a source com- 
munication apparatus, a destination communication apparatus, a plurality of transmission paths capable of being 
set between said source communication apparatus and said destination communication apparatus, and at least 
one intermediate communication apparatus each Interposed in any one of said transmission paths, said transmis- 
sion path controlling apparatus being provided in one of said communication apparatus, comprising: 

a traffic characteristic collection section for collecting traffic characteristics of those of said transmission paths 
connected to the one communication apparatus or the other communication apparatus, 
a traffic characteristic notification section for notifying the other communication apparatus of the traffic char- 
acteristics collected by said traffic characteristic collection section, 

a load calculation section for calculating load based on the traffic characteristics collected by said traffic char- 
acteristic collection section, 

a decision section for deciding whether or not a transmission path should be added or deleted based on the 
load information determined by said load calculation section, and 

a load equalization section for equalizing the load determined by said load calculation section among said 
plurality of transmission paths. 

2. A transmission path controlling apparatus for use with an internet protocol network which includes a source com- 
munication apparatus, a deslination communication apparatus, a plurality of transmission paths capable of being 
set between said source communication apparatus and said destination communication apparatus, and at least 
one intermediate communication apparatus each interposed in any one of said transmission paths, said transmis- 
sion path controlling apparatus being provided in said source communication apparatus, comprising: 

a traffic characteristic collection section for collecting traffic characteristics of those of said transmission paths 

connected to said source communication apparatus or the other communication apparatus. 

a load calculation section for calculating load based on the traffic characteristics collected by said traffic char- 
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acteristlc collection section, 

a decision section for deciding addition or deletion of a transmission path based on the load information de- 
termined by said load calculation section, and 

a toad equalization section for equalizing the load determined by said load calculation section among said 
plurality of transmission paths. 

3. A transmission path controlling apparatus for use with an Internet protocol network which Includes a source com- 
munication apparatus, a destination communication apparatus, a plurality of transmission paths capable of being 
set between said source communication apparatus and said destination communication apparatus, and at least 
one intermediate communication apparatus each interposed in any one of said transmission paths, said transmis- 
sion path controlling apparatus being provided in any one of the communication apparatus other than said source 
communication apparatus, comprising: 

a traffic characteristic collection section for collecting traffic characteristic of the transmission paths connected 
to the communication apparatus, and 

a traffic characteristic notification section for notifying said source communication apparatus of the traffic char- 
acteristlc collected by said traffic characteristic collection section. 

4 The transmission path controlling apparatus of any one of claims 1 to 3. further comprising: 

in said traffic characteristic collection section, means for smoothing the collected traffic cbaractenstic values. 

5 The transmission path controlling apparatus of claim 4, further comprising: 

in said traffic characteristic collection section, means for calculating utilizations of the transmission paths connected 
to the communication apparatus based on the collected traffic characteristic values. 

6 The transmission path controlling apparatus of claim 5, wherein said smoothing means of said traffic characteristic 
collection section Is constructed so as to smooth the utilizations calculated by said utilization calculating means. 

7 The transmission path controlling apparatus of claim 4, wherein said smoothing means of said traffic characteristic 
collection section is constructed so as to smooth the collected traffic characteristic values, and said traffic charac- 
teristic collection section has means for calculating utilizations of the transmission paths connected to the com- 
munication apparatus based on the traffic characteristic values smoothed by said smoothing means. 

8 The transmission path controlling apparatus of any one of claims 1 to 3 f further comprising: 

in said traffic characteristic collection section, means for determining a maximum value of the collected traffic 
characteristic values as a representative value of the traffic characteristic values. 

9. The transmission path controlling apparatus of claim 1 or 3, further comprising: 

in said traffic characteristic notification section, means for notifying of information regarding the traffic char- 
acteristics making use of a packet provided by a routing protocol. 

1 0. The transmission path controlling apparatus of any one of claims 1 , 3 and 9. further comprising: 

in said traffic characteristic notification section, means for notifying of information regarding the traffic char- 
acteristics at periodical intervals. 

11. The transmission path controlling apparatus of claim 1 or 3, further comprising: 

in said traffic characteristic notification section, means for notifying of information regarding the traffic char- 
acteristic with a message of the RSVP expanded. 

1 2. The transmission path controlling apparatus of any one of claims 1 to 3, further comprising: 

in said traffic characteristic collection section, means for collecting a unit of a flux of flows for collecting traffic 
characteristics in a unit of a label which is used by a communication apparatus of the label switch type. 

13. The transmission path controlling apparatus of claim 1 or 2, further comprising: 

in said traffic characteristic collection section provided in said source communication apparatus, means for 
calculating a utilization of each of the transmission paths connected to said source communication apparatus 
based on an average utilization of the transmission paths connected to said Intermediate commun.catlon apparatus 
collected from said Intermediate communication apparatus. 
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14. The transmission path controlling apparatus of claim 13, further comprising: 

in said traffic characteristic collection section provided fn said source communication apparatus, means for 
calculating a utilization of each of the transmission paths connected to said source communication apparatus 
based on a maximum utilization of the transmission paths connected to said intermediate communication apparatus 
collected from said intermediate communication apparatus. 

15. The transmission path controlling apparatus of claim 1 or 2, further comprising: 

In said traffic characteristic collection section provided in said source communication apparatus, means for 
discriminating at periodical intervals whether or not each of said transmission paths is congested. 

16. The transmission path controlling apparatus of claim 1 or 2, further comprising: 

in said load calculation section provided in said source communication apparatus, means for calculating 
effective load taking a number of abandoned packets generated in said intermediate communication apparatus 
into consideration. 

17. The transmission path controlling apparatus of claim 16 r wherein said effective load calculation means is con- 
structed so as to set an upper limit to the effective load when the effective load is to be calculated. 

18. The transmission path controlling apparatus of claim 1 or 2, further comprising: 



In said load calculation section provided in said source communication apparatus, means for calculating ef- 
fective load taking a number of abandoned packages generated in said intermediate communication apparatus 
into consideration, and 

said decision section provided in said source communication apparatus has means for calculating an effective 
25 bandwidth from the effective load obtained by said effective load calculation means and for determining, re- 

garding all of said transmission paths as one virtual pipe, a utilization of the virtual pipe. 

19. The transmission path controlling apparatus of claim 1 or 2, further comprising: 

3 o in said load calculation section provided in said source communication apparatus, means for calculating ef- 

fective load taking a number of abandoned packages generated in said intermediate communication apparatus 
into consideration, and 

said decision section provided in said source communication apparatus having 

pipe utilization calculation means for calculating an effective bandwidth from the effective load calculated by 
35 said effective toad calculation means and for determining, regarding all of said transmission paths as one 

virtual pipe, a utilization of the virtual pipe, and 

means for comparing the pipe utilization obtained by said pipe utilization calculation means with a set reference 
value and for deciding whether a transmission path is to be added or deleted in accordance with a result of 
the comparison. 



20. The transmission path controlling apparatus of claim 19, wherein 

said calculation means of said decision section Is constructed so as to calculate at least one of the pipe utili- 
zation, a rate of change of the pipe utilization per time and an integrated amount of the pipe utilization in a 
45 fixed time, and 

said transmission path addition-deletion decision means is constructed so as to perform decision of whether 
or not a transmission path should be added or deleted based on at least one of the pipe utilization, rate of 
change and Integrated amount obtained by said calculation means. 

50 21. The transmission path controlling apparatus of daim 20, wherein said decision section provided In said source 
communication apparatus is constructed so as to perform comparison of at feast one of a set of the pipe utilization 
obtained by said calculation means and a first reference value, another set of the rate of change obtained by said 
calculation means and a second reference value and a further set of the integrated value obtained by said calcu- 
lation means and a third reference value, and perform decision of whether or not a transmission path should be 

55 added or deleted in response to a result of the comparison. 

22. The transmission path controlling apparatus of claim 21, wherein said decision section provided in said source 
communication apparatus is constructed so as to add a transmission path when the pipe utilization obtained by 
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said pipe utilization calculation means is higher than a reference value for addition of a path. 

23. The transmission path controlling apparatus of claim 21, wherein said decision section provided in said source 
communication apparatus Is constructed so as to delete a transmission path having a low load when the pipe 
utilization obtained by said pipe utilization calculation means is lower than a reference value for deletion of a path. 

24. The transmission path controlling apparatus of any one of claims 19 to 23, wherein said calculation means is 
constructed so as to calculate at least one of a difference value between two pipe utilizations obtained after a very 
small interval and a differential value of the pipe utilization at a certain point of time as the rate of change of the 
pipe utilization per time. 

25 The transmission path controlling apparatus of any one of claims 19 to 23, wherein said calculation means is 
constructed so as to calculate at least one of a number of times by which the pipe utilization exceeds a certain 
reference value within a fixed period of time and an integral value of the pipe utilization within the fixed time as the 
integrated amount of the pipe utilization within the fixed period of time. 

26. The transmission path controlling apparatus of claim 1 or 2, wherein 

said decision section provided in said source communication apparatus includes 

means for selecting a candidate of a transmission path as an object of deletion based on the load information 
determined by said load calculation section, 

means for estimating the load to the other transmission paths when the transmission path of the candidate of 
deletion selected by said deletion candidate selection means is deleted, and 

means for comparing the load to the other transmission paths estimated by said load estimation means with 
a predetermined reference value and determining whether or not the transmission path of the candidate of 
deletion should be deleted in response to a result of the comparison. 

27. The transmission path controlling apparatus of claim 26, further comprising: 

in said load calculation section provided in said source communication apparatus, means for calculating an 

effective load taking a number of abandoned packets generated in said intermediate communication apparatus 

into consideration, and • 

said decision section provided In said source communication apparatus further includes 

calculation means for calculating an effective bandwidth from the effective load obtained by said effective load 

calculation means and determining, regarding all of said transmission paths as one virtual pipe, information 

regarding a utilization of the virtual pipe, and 

trigger means for triggering said selection means, estimation means and determination means to operate 
based on the information regarding the pipe utilization obtained by said calculation means. 

28. The transmission path controlling apparatus of claim 1 or 2. further comprising: 

in said load equalization section provided in said source communication apparatus, move effective bandwidth 
calculation means for determining an effective bandwidth to be moved with each of said transmission paths from 
an average effective bandwidth calculated for all of said transmission paths based on an effective bandwidth ob- 
tained from effective load obtained taking a number of abandoned packets generated in said intermediate com- 
munication apparatus into consideration. 

29 The transmission path controlling apparatus of claim 28. further comprising: 

in said load equalization section provided m said source communication apparatus, means for distributing 
packets flowing Into said source communication apparatus to said plurality of transmission paths m proportion to 
the effective bandwldths to be moved calculated by said move effective band calculation means. 

30. The transmission path controlling apparatus of claim 29, wherein said load equalization sectl on pro vided in said 
source communication apparatus Is constructed so as to perform a calculation using a Hash function based on an 
address and distribute traffic flows at random based on a result of the calculation. 

31. The transmission path controlling apparatus of claim 30, wherein said load equalization section provide >d in ^said 
source communication apparatus performs the calculation using a CRC (cyclic redundancy check) as the Hash 
function. 
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32 The transmission path controlling apparatus of claim 30. wherein said load equalization section provided in said 
source communication apparatus performs at least one of replacement of posiUons and logical operebon for bit 
values of an input value as the calculation by the Hash function to produce a random integer value. 

33. The transmission path controlling apparatus of claim 30. wherein said load equalization section inriM tiNU 
source communication apparatus additionally uses, as an input value fortha Hash function, a control value other 
than the address Included In each packet. 

34. A transmission path controlling method for an internet protocol network which includes 8 «^^ u ^^ 
apparatus, a destination communication apparatus, a plurality of transmission paths capable of being set between 
said source communication apparatus and said destination communication apparatus, and at feast one .nterme- 
diate communication apparatus each interposed in any one of said transmission paths, the method comprising: 

(a) collecting traffic characteristics of those of said transmission paths connected to other communication 

(bf^^ai source communication apparatus of the traffic characteristics collected by step (a), 

steps (a) and (b) being performed by a communication apparatus other than said source commun.cat.on 
apparatus of said internet protocol network, and 

(c) collecting traffic characteristics of those of said transmission paths connected to said source communication 

fdTcaTcufating load based on one or both of the traffic characteristics collected by the traffic characteristic 
collection step and the traffic characteristics obtained from the other communication apparatus. 

(e) deciding whether or not a transmission path should be added or deleted based on the load information 
determined by the load calculation step, and , . 

(f) equalizing the load determined by the load calculation step among said plurality of transmission paths. 

steps (d), (e), and (0 being performed by said source communication apparatus. 

35 A medium on which a transmission path controlling program for use with an internet protocol network which includes 
" a source communication apparatus, a destination communication apparatus, a plurality of transmission paths ca- 
pable of being set between said source communication apparatus and said destination commun.cation apparatus, 
and at least one intermediate communication apparatus each interposed in any one of said transmission paths is 
recorded, said transmission path controlling program being used by one of said communication apparatus, said 
transmission path controlling program causing a computer to function as 

traffic characteristic collection means for collecting traffic characteristics of those of said transmission paths 
connected to the communication apparatus or the other communication apparatus, 

traffic characteristic notification means for notifying the other communication apparatus of the traffic charac- 
teristics collected by said traffic characteristic collection means. 

load calculation means for calculating load based on the traffic characteristics collected by said traffic char- 
acteristic collection means, 

decision means for deciding whether or not a transmission path should be added or deleted based on the load 
information determined by said load calculation means, and 

load equalization means for equalizing the load determined by said load calculation means among said plurality 
of transmission paths. 

36 A network load balancing apparatus which balances the traffic load among transmission paths set between an 
* ingress node and an egress node in a network having a plurality of network nodes, said network having a plurality 

of transmission paths capable of being set between said ingress node and said egress node, comprising: 

a traffic characteristic collection section for determining traffic characteristics of said transmission paths set 
between said ingress node and said egress node, 

a traffic characteristic notification section for notifying network nodes of results of said traffic charactenst.c 
collection section, ■ . 

a load calculation section for calculating traffic load based on the traffic charactensbcs collected by said traffic 
characteristic collection section, and 

a means for causing transmission of a packet on a transmission path set between said ingress node and said 
egress node depending upon said traffic load calculation. 
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37 The network load balancing apparatus of claim 36, further comprising: 

a decision section for deciding whether or not a transmission path should be added or deleted between said 
Ingress node and said egress node based on the traffic load information determined by said load calculation section. 

38 The network load balancing apparatus of claim 36, further comprising: 

a load equalization section for equalizing the load determined by said load calculation section among said 
set transmission paths. 

39 The network load balancing apparatus of claim 36, further comprising: 

means in said In traffic characteristic notification section for providing said results of the Informatfon regarding 
the iraffic characteristics periodically and by making use of a packet provided by a routing protocol. 

40. The network load balancing apparatus of claim 36. wherein 

said traffic characteristic collection section calculates occupancy of the set transmission paths based on the 
collected traffic characteristic values. 

41. The network load balancing apparatus of claim 40. wherein 

said traffic characteristic collection section smooths the calculated occupancy of the set transmission paths. 

42 The network load balancing apparatus of claim 40, wherein • 

said traffic characteristic collection section smooths the collected traffice characteristic values, and said traffic 
characteristic collection section calculates occupancy of the set transmission paths based on the smoothed col- 
lected traffic characteristic values. 

43 The network load balancing apparatus of claim 36, further comprising: 

means in said in traffic characteristic notification section for collecting a unit of flux of flows for thereby col- 
lecting traffic characteristics in a unit of a label utilized by a router of a label switch type. 

44 A network load balancing apparatus which balances the traffic load among transmission links which link node 
JevSTs 5 a network, and where at least one transmission link defines a transmission path between an .Ingress 
node and an egress node, and said network having a plurality of transmission paths capable of being set between 
said ingress node and said egress node, comprising: 

a traffic characteristic collection section for collecting traffic characteristics of said transmission links being 

portions of transmission paths set between said ingress node and said egress node, 

a traffic characteristic notification section for providing results of said traffic characteristic collection sectic^ 

a load calculation secUon for calculating traffic load based on the traffic characteristics collected by said traffic 

characteristic collection section, and . ^ 

a means for causing transmission of a packet on a transmission path set between said ingress node and said 

egress node depending upon said traffic load calculation. 

45. The network load balancing apparatus of claim 44, further comprising: 

a decision section for deciding whether or not a transmission path should be added or deleted between sa.d 
ingress node and said egress node based on the load information determined by said load calculate section. 

ag The network load balancing apparatus of claim 44, further comprising: 

46. The netwo * Jjjj^ ^ ^ ^ by ^ |oa<J section among sa.d 
set transmission paths. 

si The network load balancing apparatus of claim 44, further comprising: 

a meansprovided in said traffic characteristic collection section for calculating an occupancy of each trans- 
mission path among said set transmission paths based on an average occupancy of transm,ss.on l.nks compns.ng 
said each transmission path. 

48 The network load balancing apparatus of claim 44, further comprising: . • 

a means provided in said traffic characteristic collection section for calcu.at.ng an occupancy of e^ans- 
mlsslon path among said set transmission paths based on a maximum occupancy of transn.os.on l.nks compns.ng 
said each transmission path. 
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49 A transmission path controlling apparatus for use with an internet protocol network which includes a source router, 
' a destination router, a plurality of transmission paths capable of being set between said source router and se.d 

destination router, said internet protocol network having a plurality of network devices including said source and 
destination routers, said transmission path controlling apparatus comprising: 

a traffic characteristic collection section for collecting traffic characteristics of those of said transmission paths 
set between said source router and said destination router, 

a traffic characteristic notification section for notifying network devices of results from said traffic charactenstic 

collection section, ^ f M , , ^ ._, . _ -. r 

a load calculation section for calculating load based on the traffic characteristics collected by said traffic char- 
acteristic collection section, and 

a means for causing transmission of a packet on a transmission path set between said source router and said 
destination router depending upon said traffic load calculation. 

50 The transmission path controlling apparatus of claim 49. further comprising: 

a decision section for deciding whether or not a transmission path should be added or deleted based on the 
load determined by said load calculation section. 

51 The transmission path controlling apparatus of claim 49. further comprising: 

a load equalization section for equalizing the load determined by said load calculation section among said 
set transmission paths. 

52. The transmission path controlling apparatus of claim 4g. further comprising: 
means for smoothing the collected traffic characteristics. 

53 The transmission path controlling apparatus of claim 52, wherein said traffic characteristic collection section cal- 
' culates the occupancy of those of said transmission paths set between said source router and said destination 

router based on the smoothed traffic characteristics. 

54 The transmission path controlling apparatus of claim 49, further comprising: 

means for determining a maximum value of the collected traffic characteristics as a representative value of 
the traffic characteristic values. 

55 The transmission path controlling apparatus of claim 49. further comprising: 

means for periodically notifying information regarding the traffic characteristics making use of a packet pro- 
vided by a routing protocol. 

56 The transmission path controlling apparatus of claim 49. further comprising: 

means for notifying information regarding the traffic characteristic with a message of RSVP expanded pro- 
tocol. 

57. The transmission path controlling apparatus of claim 49. further comprising: 

means for collecting a unit of a flux of flows thereby collecting traffic characteristics in a unit of a label which 
is used by a router of a label switch type. 

58. The transmission path controlling apparatus of ciaim 49, further comprising: 

means for calculating an occupancy of each transmission path connected to said source router based on an 
average occupancy of the transmission paths connected to an intermediate router collected from said intermediate 
router, wherein said intermediate router may be interposed in any. one of said transmission paths. 

59. The transmission path controlling apparatus of claim 49, further comprising: 

means for calculating an occupancy of each of the transmission paths connected to said source router based 
on a maximum occupancy of the transmission paths connected to an intermediate router collected from said in- 
termediate router, wherein said intermediate router may be interposed in any one of said transmission paths. 

60. The transmission path controlling apparatus of claim 49, further comprising: 

means for determining periodically whether or not each of said transmission paths is congested. 
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61. The transmission path controlling apparatus of claim 49. further comprising: 

means provided in said load calculation section for calculating effective load taking a number of abandoned 
packets generated in an Intermediate router into consideration, wherein said Intermediate router may be interposed 
in any one of said transmission paths. 

5 

62. The transmission path controlling apparatus of claim 61 , wherein 

said means is constructed so as to set an upper limit to the effective load when the effective load is to be 
calculated. 

10 63. The transmission path controlling apparatus of claim 50. further comprising: 

means provided In said load calculation section for calculating effective load taking a number of abandoned 
packages generated in an intermediate router into consideration, and 

means provided in said decision section for calculating an effective bandwidth from the effective load obtained 
75 by said effective load calculation means and determining, regarding all of said transmission paths as -one 

virtual pipe, an occupancy of tha virtual pipe. 

64. The transmission path controlling apparatus of claim 50. further comprising: 

20 means for calculating effective load by considering a number of abandoned packages generated in an inter- 

mediate router, and 

said decision section having a pipe utilization calculation unit for calculating an effective bandwidth from the 
effective load calculated by said effective load calculation means and determining, regarding all of said trans- 
mission paths as one virtual pipe, a utilization of the virtual pipe, and 
25 means for comparing the pipe utilization obtained by said pipe utilization calculation unit with a set reference 

value and determining whether a transmission path is to be added or deleted in accordance with a result of 
the comparison. 

65. The transmission path controlling apparatus of claim 64. wherein said decision section is constructed so as to add 
30 a transmission path when the pipe utilization calculation obtained by said pipe utilization calculation unit is higher 

than a reference value for addition of a path. 

66 The transmission path controlling apparatus of claim 64. wherein said decision section is constructed so as to 
delete a transmission path having a low load when the pipe utilization obtained by said pipe utilization calculation 
35 unit is lower than a reference value for deletion of a path. 

67. The transmission path controlling apparatus of claim 51, further comprising: 

means for determining an effective bandwidth to be moved among each of said transmission paths from an 
average effective bandwidth calculated for all of said transmission paths based on an effective bandwidth obtained 
40 from effective load obtained taking a number of abandoned packets generated in an intermediate router into con- 

sideration. 

68. The transmission path controlling apparatus of claim 67. further comprising: 

means for distributing packets flowing into said source router to said plurality of transmission paths in pro- 
«s portion to the effective bandwidths to be moved calculated by said move effective band calculation means. 

69 The transmission path controlling apparatus of claim 68, wherein said load equalization section performs a calcu- 
lation using a hash function based on an address and distributes traffic flows at random based on a result of the 
calculation. 

50 

70 A transmission path controlling method for a network which includes a source router, a destination router, and a 
plurality of transmission paths capable of being set between said source router and said destination router, com- 
prising the steps of: 

55 collecting traffic characteristics of transmission paths set between said source router and said destination 

calculating load of transmission paths set between said source router and said destination router based resutts 
of the traffic characteristics collection step, and 
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49 A transmission path controlling apparatus for use with an internet protocol network which includes a source router, 
a destination router, a plurality of transmission paths capable of being set between said source router and said 
destination router, said internet protocol network having a plurality of network devices including said source and 
destination routers, said transmission pain controlling apparatus comprising: 

a traffic characteristic collection section for collecting traffic characteristics of those of said transmission paths 
set between said source router and said destination router, 

a traffic characteristic notification section for notifying network devices of results from said traffic characterise 
collection section, , 
w a load calculation section for calculating load based on the traffic characteristics collected by sa.d traffic char- 

acteristic collection section, and 

a means for causing transmission of a packel on a transmission path set between said source router and said 
destination router depending upon said traffic load calculation. 

15 50 The transmission path controlling apparatus of claim 49. further comprising: 

a decision section for deciding whether or not a transmission path should be added or deleted based on the 
load determined by said load calculation section. 

51 The transmission path controlling apparatus of claim 49, further comprising: 
20 a load equalization section for equalizing the load determined by said load calculation section among said 

set transmission paths. 

52. The transmission path controlling apparatus of claim 4g. further comprising: 
meens for smoothing the collected traffic characteristics. 

53 The transmission path controlling apparatus of claim 52, wherein said traffic characteristic collection section cal- 
culates the occupancy of those of said transmission paths set between said source router and said destination 
router based on the smoothed traffic characteristics. 

30 54. The transmission path controlling apparatus of claim 49, further comprising: 

means for determining a maximum value of the collected traffic characteristics as a representative value of 
the traffic characteristic values. 

55. The transmission path controlling apparatus of claim 49. further comprising: 

3S means for periodically notifying information regarding the traffic characteristics making use of a packet pro- 

vided by a routing protocol. 

56. The transmission path controlling apparatus of claim 49. further comprising: 

means for notifying information regarding the traffic characteristic with a message of RSVP expanded pro- 

40 tQCOl. 

57. The transmission path controlling apparatus of claim 49. further comprising: 

means for collecting a unit of a flux of flows thereby collecting traffic characteristics in a unit of a label which 
is used by a router of a label switch type. 

58 The transmission path controlling apparatus of claim 49. further comprising: 

means for calculating an occupancy of each transmission path connected to said source router based on an 
average occupancy of the transmission paths connected to an intermediate router collected from said intermediate 
router, wherein said intermediate router may be interposed in any one of said transmission paths. 

59. The transmission path controlling apparatus of claim 49. further comprising: 
means for calculating an occupancy of each of the transmission paths connected to said source router based 

on a maximum occupancy of the transmission paths connected to an intermediate router collected from said in- 
termediate router, wherein said intermediate router may be interposed in any one of said transmission paths. 

60. The transmission path controlling apparatus of claim 49, further comprising: 
means for determining periodically whether or not each of said transmission paths is congested. 
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61. The transmission path controlling apparatus of claim 49. further comprising: 

means provided in said load calculation section for calculating effective load taking a number of abandoned 
packets generated in an intermediate router into consideration, wherein sard intermediate router may be interposed 
in any one of said transmission paths. 

5 

62. The transmission path controlling apparatus of claim 61. wherein 

said means is constructed so as to set an upper limit to the effective load when the effective load is to be 
calculated. 

10 63. The transmission path controlling apparatus of claim 50, further comprising: 

means provided In said load calculation section for calculating effective load taking a number of abandoned 
packages generated in an intermediate router into consideration, and 

means provided in said decision section for calculating an effective bandwidth from the effective load obtained 
t5 by said effective load calculation means and determining, regarding all of said transmission paths as one 

virtual pipe, an occupancy of trie virtual pipe. 

64. The transmission path controlling apparatus of claim 50. further comprising: 

20 means for calculating effective load by considering a number of abandoned packages generated in an Inter- 

mediate router, and 

said decision section having a pipe utilization calculation unit for calculating an effective bandwidth from the 
effective load calculated by said effective load calculation means and determining, regarding all of said trans- 
mission paths as one virtual pipe, a utilization of the virtual pipe, and 
25 means for comparing the pipe utilization obtained by said pipe utilization calculation unit with a set reference 

value and determining whether a transmission path is to be added or deleted in accordance with a result of 
the comparison. 

65. The transmission path controlling apparatus of claim 64, wherein said decision section is constructed so as to add 
30 a transmission path when the pipe utilization calculation obtained by said pipe utilization calculation unit is higher 

than a reference value for addition of a path. 

66 The transmission path controlling apparatus of claim 64, wherein said decision section is constructed so as to 
delete a transmission path having a low load when the pipe utilization obtained by said pipe utilization calculation 
35 unit is lower than a reference value for deletion of a path. 

67. The transmission path controlling apparatus of claim 51, further comprising: 

means for determining an effective bandwidth to be moved among each of said transmission paths from an 
average effective bandwidth calculated for all of said transmission paths based on an effective bandwidth obtained 
40 from effective load obtained taking a number of abandoned packets generated in an intermediate router into con- 

sideration. 

68. The transmission path controlling apparatus of claim 67. further comprising: 

means for distributing packets flowing into said source router to said plurality of transmission paths in pro* 
45 portion to the effective bandwldths to be moved calculated by said move effective band calculation means. 

69. The transmission path controlling apparatus of claim 68. wherein said load equalization section performs a calcu- 
lation using a hash function based on an address and distributes traffic flows at random based on a result of the 
calculation. 

70. A transmission path controlling method for a network which includes a source router, a destination router, and a 
plurality of transmission paths capable of being set between said source router and said destination router, com- 
prising the steps of: 

55 collecting traffic characteristics of transmission paths set between said source router and said destination 

router, 

calculating load of transmission paths set between said source router and said destination router based results 
of the traffic characteristics collection step, and 
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transmitting a packet on a transmission path set between said ingress node and said egress node depending 
upon said traffic load calculation. 

71. The transmission path controlling method of claim 70, further comprising the stop of; 

determining whether or not a transmission path should be added or deleted based on the load information 
determined by the load calculation step. 

72. The transmission path controlling method of claim 70. further comprising the step of: 

equalizing the load determined by the load calculation step among said plurality of transmission paths. 

73. The transmission path controlling method of claim 70. further comprising the step of. 

providing the resufts of the traffic characteristic collection step to node devices of the network. 

74. A transmission path controlling method for an internet protocol network which includes a source router, a destination 
router, a plurality of transmission paths capable of being set between said source router and satd destination router, 
and an intermediate router interposed in any one of said transmission paths, comprising the steps of: 

collecting traffic characteristics of transmission paths connected to at least one of intermediate router and the 
destination router, and 

notifying said source router of the results of the traffic characteristics collection step. 

75. A computer-readable medium having computer executable instructions for use in an internet protocol network 
which includes a source router, a destination router, a plurality of transmission paths capable of being set between 
said source router and said destination router, said computer executable instructions being used by a router, and 
for performing the steps comprising: 

collecting traffic characteristics of transmission paths set between said source router and said destination 
router, and 

notifying other routers of the traffic characteristics collected. 

76. A computer-readable medium of claim 75. having computer executable instructions for performing the further Steps 
comprising: 

calculating load based on Ihe traffic characteristics collected, and 
transmitting a packet based on the results of the calculated load. 

77. A computer-readable medium of claim 75. having computer executable instructions for performing the further steps 
comprising: . 

determining whether or not a transmission path should be added or deleted based on the load information 
determined by said load calculation means, and 

load equalization means for equalizing the load determined by said load calculation means among said plurality 
of transmission paths. 

78. A label switch router In a Multi-Protocol Label Switch network, comprising: 

a traffic characteristic collection section for collecting for each port thereof, traffic characteristic values Including 
an output packet amount from an output port and a logical bandwidth of said output port as traffic characteristic 
values; and 

a traffic characteristic notification section for notifying network nodes of results from said traffic characteristic 
collection section. 

79. The label switch router of claim 76, further comprising: 

means for processing the obtained traffic characteristic values by calculating a transmission line utilization 
value CUTY from the obtained traffic characteristic values In accordance with the following expression 
CUTY = output packet amount from an output port/logical bandwidth of said output port. 
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80. The label switch router of claim 79, further comprising: 

means for processing the obtained traffic characteristic values by performing smoothing of the collected 
traffic characteristic values. 

5 81. The label switch router of claim 80, wherein said smoothing processing is performed by calculating a value MUTY 
each time an observation value is obtained, wherein 

MUTY = a X CUTY + (1 - a) x MUTY 

10 

where a is a smoothing coefficient. 

82. The label switch router of claim 81. further comprising: 

means for determining a maximum value of collected characteristic values as a representative value of the 
is traffic characteristic values, wherein the maximum value of the observation values is placed as a working value 

MUTY. 

83. The label switch router of claim 82, wherein said traffic characteristic notification section distributes said traffic 
characteristic values including MUTY periodically utilizing a routing protocol. 

20 

84. The label switch router of claim 78. wherein said traffic characteristic values including an abandoned packet number 
NLOSS at said output port In a time period, said label switch router further comprising: 

means for processing the obtained traffic characteristic values by calculating a total number TLOSS of aban- 
25 doned packets in accordance with the following expression. 

TLOSS(n) = TLOSS(n-l) + NLOSS 

30 where TLOSS(n) Is the total number of abandoned packets during time period n, TLOSS(n-l) is the total 

number of abandoned packets in a prior time period. 

85. The label switch router of claim 84. wherein said traffic characteristic notification section distributes said traffic 
characteristic values including TLOSS(n) periodically utilizing a routing protocol. 

35 

86. The label switch router of claim 85, wherein said distribution utilizes a message of an RSVP expanded protocol to 
notify another label switch router of the Information. 

87. The label switch router of claim 78, further comprising: 

ao a traffic characteristic collection section for receiving traffic characteristics from another label switch router. 

88. The label switch router of claim 81 , further comprising: 

a traffic characteristic collection section for receiving traffic characteristics from another label switch router, 
45 a load calculation section for calculating traffic load based on at least one of traffic characteristics collected 

from said traffic characteristic collection section and received traffic characteristics from another label switch 
router, and 

a packet transfer section for transferring a packet based on the results of said load calculation section. 

so 89. The label switch router of claim 88, wherein 

said load calculation section further calculates a congestion state of individual paths from said label switch 
router to another label switch router where a plurality of paths are set between said label switch router lo said 
another label switch router, and each path may extend through a plurality of repeating label switch routers and 
said path is composed of links between said repealing label switch routers, said links connecting said repeating 

55 label switch routers to form said path, the congestion state for each path is determined, from an occupancy of 

each path, collected from each of said repeating label switch routers of each path, based on an average occupancy 
of each link connected to each repeating label switch router. 
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90. The label switch router of claim 89. wherein 

the occupancy of a link J of a path i are averaged and may represented In such an expression as 

p path i = Average(MUTY(link j, path I). 
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91. The label switch router of claim 90, wherein 

a maximum occupancy of each link connected to each repeating label switch router is calculated where a 
maximum value of occupancy of all links of a path I Is adopted and may be represented in the expression 

path i = Max(MUTY(link j, path i). 



92. The label switch router of claim 84, further comprising: 

a traffic characteristic collection section for receiving traffic characteristics from another label switch router, 
a load calculation section for calculating traffic load based on at least one of traffic characteristics collected 
from said traffic characteristic collection section and received traffic characteristics from another label switch 
router, and _ 
a packet transfer section for transferring a packet based on the results of said load calculation section. 

93. The label switch router of claim 92, wherein 

a plurality of paths are set between said label switch router to another label switch router, and each route 
extends through a plurality of repeating label switch routers and is composed of links between said repeating label 
25 switch routers, the congestion state for each route is determined from an occupancy of each path, collected from 

each of said repeating label switch routers of each path, based on an average occupancy of each link connected 
to each repeating label switch router. 

94. The label switch router of claim 93, wherein 

30 a sum total TUOSS path i of abandoned packets of said path i is determined by the expression 

TLOSS path I = ETLOSS link j 

35 where i is the number of a label switch path and j is a link number. 

95. The label switch router of claim 94, wherein 

calculation of an effective load p effective of path i is calculated by the expression 



p effective path i = ppath I x f (TLOSS path I) 

where p path i is the load of path i and f(TLOSS path i) is a function regarding TLOSS path I. 

45 96. The label switch router of claim 95» wherein 

if packet loss occurs, then the load p path i is corrected with a predetermined function f(TLOSS path I) ac- 
cording to the expression 

p effective path i = Min( p effective path i, p ceiling) 

so 

where f (TLOSS path i) is a function regarding TLOSS path i, and p ceiling is an upper limit load value to the 
effective load p effective path i. 

55 97. The label switch router of claim 94, wherein each route extends through a plurality of repeating label switch routers, 
load based on traffic characteristics may be calculated utilizing a number TLOSS path i of abandoned packets 
generated in each repeating label switch router. 
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the effective load p effective path i in accordance with 
effective road to Ith path ■ toad to ilh path x ((abandoned packer number) 

where f(abandoned packet number) is a function wherein the abandoned packet number is a variable. 

98. The label switch router of claim 94, further comprising: 

a load equalization section for adjusting the load periodically where all paths are regarded as a single virtual 
pipe, and a load to said virtual pipe is determined by means for calculating effective bandwidths from effective 
loads obtained by said effective load calculation section and determining a utilization of the virtual pipe. 

99. The label switch router of claim 98, wherein 

said utilization of the virtual pipe is determined in accordance with the expression 

utilization p effective all when alt paths are regarded as one virtual pipe = X ( p effective path i x LBW 
path 0/ELBW path f 

where p effective path Is the effective load of the path i. and LBW path i is the logical bandwidth of the path i. 



100 The label switch router of claim 99, wherein 

said utilization of the virtual pipe is compared with a set reference value p offerl, and when the obtained 
25 value of the utilization is higher than P offerl , a new transmission path is added. 

1 01 -The label switch router of claim 99, wherein 

said utilization of the virtual pipe is compared with a set reference value p offer2, and if the obtained value 
of the utilization is lower than p offer2, then a path is deleted. 

30. 



35 



40 



45 



SO 



55 



33 



EP1 130 849 A2 



FIG. 1(a) 



1R 1R 




2-N 



34 



BP 1 130 849 A2 



FIG.l(b) 



A- 



Decision 
section 



Load calculation 
section 



Trarflc characteristics 
to other routers 



EE 



Traffic characteristic 
notification section 



I 



Load equalization 
section 



Transmission 
^ line 

* selection » 



3B 



rrartlc characteristic 
rccepUon seeuon 



rraffic eharactaristii 
collection section 



y 



3A 



Traffic characteristics 
from other routers 



Traffic characteristics 
of own links 



FIG.l(c) 



Traffic characteristics 
to other routers 



Traffic characterise 
notification section 



rraffic characteristic 
reception section 



38 



rraffic characteristic 
coOectfon section 



y 



3A 



"J 



Traffic characteristics 
from other routers 



Traffic characteristics 
of own links 



35 



EP 1 130 849 A2 



FIG.2(a) 



Core LSR 
(Label Switching Router) 



Edge LSR 1 1 C 



t 1 E 



1 J C E<i 9« LSR 



90 



1 1 E 





















— > 









FIG.2(b) 





Label A 




Label B 




Label C 


IP header . 




IP header 


tP header 


IP header 


IP packet 




IP packet 


IP packet 


IP packet 









IP header 
IP packet 



36 



EP1 130 849 A2 

FIG.3 



Label stack 

\ label [gg[mf \ 

. ■ \ 
[label |S=0|tTL] 

[ label IS^lfrtV ' 



FIG.4 



Label (20b its) 


Reserved 


S 


TT L 


Obits) 


Obit) 


(8b its) 



4 octal* 



FIG.5 




LAN (Ether, ale. ) 



L3 pay load 



L3 header 



Shim header 



L2 header 



X 



LLC/SNAP Header 
ethertype 

=0x8847[MPLS unlcast] 
=0x884&[MPLS multicast] 



PPP(overSONET) 



^0x828tUPLSCP] 
=0x0281 DJPLS unlcastj 
-0x0283 [MPLS multicast] 



37 



EP1 130 849 A2 




38 



EP 1 130 849 A2 



4 




39 



EP1 130 849 A2 



CO B 

— I CO 



00 

d 



E 
S 



o g 

C if 




3 

o 



u 
0- 



it. 

lis 



2 

S 
2 



CO 



3 



to 

a. 



S £ 

2 S 

5 3 



flu 



I 



§ 

if 
SI 



CS| 



2 

a 
a. 



40 



EP 1 130 849 A2 




EP 1 130 849 A2 



FIG.10 




Actual load 



FIG.11 



Path 1 



Path 2 



Palh3 



^ AEBW1 ! 



AEBW2 




Effecttva bandwidth 



42 



EP1 130 849 A2 



FIG. 12 



X(p 0ff0CtlV«X LBW) : Sum total of actual traffics 
«, » 



-+-4 



p ef fact! vox LBW 



Before load adjuatmont 


pathl | path2 j path3 


path4 




\ \ 

Ats \ 

\ \ 


\ 

\ 




After load adjustment 


pathl | path2 | paths 


path4 



p effect i vex LBW + Ats 



FIG. 13 



32b its IP address (IPv4) 
•< >- 





Upper 16b i ts Lower 16bits 



EXOR 

f 

16bits integer 



43 



EP1 130 849 A2 



FIG.14 



120R^3 120R-4 
oc3 

! |=- 

load 

IP flows balancing// \\ 0G z 
M II 1 * 3 / 120R-2 \ links 



3 

121 







L lOObase-T 









120R-1 




120R-5 



FIG.15 




time [sec] 



44 



EP 1 130 849 A2 



FIG.U 



60 | — — n 1 r 1 1 \ 1 1 m r 




0 100 200 300 



time [sec] 



45 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ black borders 

□ image cut off at top, bottom or sides 

□ faded text or drawing 

□ blurred or illegible text or drawing 

□ skewed/slanted images 

□'color or black and white photographs 

□ gray scale documents 

©Klines or marks on original document 

□ reference(s) or exhibit(s) submitted are poor quality 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



